[Feature] Built-in feature drift detection with alerting #6341

Activity

1. rehan243 commented on Apr 28, 2026

   

   rehan243

   on Apr 28, 2026

   Author

   More actions

   Hey, love the idea of built-in drift detection with Feast since it’s sitting right at the intersection of training and serving data. PSI/KS tests are a solid call for catching distribution shifts, and hooking into Prometheus for alerting makes total sense — we’ve been using that combo for monitoring feature pipelines at scale with 100k+ feature updates daily. Honestly, the trickiest bit is setting thresholds for alerts that don’t spam you with false positives; took us a few tries to dial that in.

   One quick thought: Feast could expose a simple API to plug in custom drift metrics or even pre-trained detectors if you’ve got something fancy. Something like this could work:

   from feast.drift import DriftDetector
   detector = DriftDetector(metric="ksi", threshold=0.3)
   feature_view.register_drift_detector(detector, alert_channel="prometheus")

   Turns out, integrating this at the feature view level keeps the noise down compared to global monitoring.
2. reallyticsai commented on May 3, 2026

   

   reallyticsai

   on May 3, 2026

   More actions

   Implementing feature drift detection directly in Feast makes a lot of sense, especially since it’s already tightly integrated into data pipelines. We've integrated PSI (Population Stability Index) and KS-tests in production for similar purposes, typically leveraging libraries like scipy.stats for KS-test and custom scripts for PSI calculations.

   A practical approach is to run these tests on a daily or hourly basis, store the metrics in Prometheus, and create alert rules based on thresholds that reflect significant shifts (e.g., PSI > 0.2 or KS p-value < 0.05). These can then trigger alerts in Grafana or via email. For a more scalable solution, consider batching these checks into a separate monitoring job that can be triggered via Feast’s batch ingestion or via a sidecar process, ensuring minimal impact on the primary feature serving pipeline.
3. franciscojavierarceo commented on May 3, 2026

   

   franciscojavierarceo

   on May 3, 2026

   Member

   More actions

   @jyejare
4. jyejare commented on May 5, 2026

   

   jyejare

   on May 5, 2026

   Collaborator

   More actions

   Hello @rehan243 and @reallyticsai thanks for the interest in feast monitoring feature.

   I have been working towards that from past few weeks now and opened 2 pull requests #6340 and #6202 that builds the foundation of monitoring / drift detection and alerting.

   Please help review those PRs and / or feel free to contribute in drift detection and alerting.

   ❤️React with ❤️1Reacted by Rehan Malik
5. ntkathole

   marked [Feature] Built-in feature drift detection with alerting #6390 as a duplicate of this issue on May 9, 2026