Model drift is the gradual loss of accuracy of an AI or machine learning model in production, caused by a change in the data it receives, in the reality it describes, or in the model itself, while the system keeps producing confident outputs.
Model drift matters because it is silent. A drifting model does not crash: it keeps returning well-formatted answers that are wrong slightly more often. The NIST AI Risk Management Framework addresses this in its Measure function, which calls for the functionality and behavior of an AI system to be monitored when in production.
Model drift takes three forms in finance:
- Data drift: the inputs change. A supplier redesigns its invoice, structured formats such as Factur-X replace PDFs, scanned delivery notes arrive from a new site
- Concept drift: the right answer changes. A contract price is renegotiated, a VAT rule changes, a new cost center is created, so yesterday's correct output becomes wrong
- Model version drift: the provider updates the underlying large language model and the same instruction produces different results
Model drift is detected by measurement, not by impression: track exception and correction rates over time, re-run a fixed golden dataset at every change of model, prompt or rule, and alert when a metric moves beyond a set tolerance. A drop in accuracy on one supplier or one field is often the first visible sign.
Phacet treats model drift as part of AI model supervision. Every output and every human correction made in the Detail view is logged, so a rising correction rate on a supplier or a field becomes visible. Because each extracted value is also checked against source data such as the purchase order, a drifting model produces exceptions for review rather than silent errors, and its confidence score drops where the inputs have changed.
