Excellent offline, mediocre live, no error anywhere
A churn model scores 0.91 AUC on a held-out set built with a point-in-time-correct feature store query. Deployed behind an API, its live decisions perform at roughly the level of the previous heuristic — a large, consistent gap with the offline number.
Feature values logged at serving time are within their expected ranges. No nulls above the usual rate. The model artefact's hash matches the one evaluated offline. Input schema validation passes on every request.
Training features were computed in a Spark job over the warehouse. Serving features come from a Redis online store populated by a separate streaming job.
The artefact is identical and the inputs validate. Give me your ranked hypotheses for the offline-online gap, and for each the comparison you would run to confirm it.