Model Insight

Model insight shows feature importance at query/category level and drift detection to understand how ML models make ranking decisions.

Overview

Model insight helps you understand how machine learning models rank products by showing feature importance and detecting when models drift from expected behavior.

Feature Importance

{
  "model": "ranking_model_v2",
  "feature_importance": {
    "query_category": "running_shoes",
    "features": [
      {
        "feature": "click_through_rate",
        "importance": 0.35
      },
      {
        "feature": "conversion_rate",
        "importance": 0.30
      },
      {
        "feature": "revenue_per_impression",
        "importance": 0.20
      },
      {
        "feature": "margin",
        "importance": 0.15
      }
    ]
  }
}

Drift Detection

{
  "drift_detection": {
    "enabled": true,
    "metrics": [
      "feature_distribution",
      "prediction_distribution",
      "performance_metrics"
    ],
    "alert_threshold": 0.1
  }
}

Best Practices

1. Monitor regularly: Check feature importance regularly
2. Detect drift: Set up drift detection alerts
3. Understand context: Consider query/category context
4. Retrain when needed: Retrain models when drift detected

Related Topics

• LLM Based Ranking and Relevance - LLM ranking trained on your customer data
• Change Logs - Track changes and impact