What causes model drift?
Changes in the data, in user behavior, in your business rules, in the documents a system retrieves from, or in the model behind a provider's API. Edits to your own prompts can cause it too.
What is the difference between data drift and concept drift?
Data drift means the inputs change while the right answers stay the same. Concept drift means the right answer for the same input changes, so the relationship the model learned no longer holds.
Do LLMs drift?
The model's weights do not change on their own, but LLM applications still drift. Providers update models, your documents age, prompts get edited and users ask new things. Test regularly.
How often should I test for drift?
Run the test set on every change to a prompt, retrieval setting, tool or model, and on a regular schedule such as weekly. Raise the frequency when the feature carries more risk.
Can I prevent drift?
Not entirely. You can limit it by pinning model versions, keeping prompts in version control, assigning owners to data and documents, and running scheduled tests so drift is found early.
Is model drift the same as model decay?
They are close. Drift is the change in conditions, and decay or degradation is the resulting loss in quality. In practice people use the terms together.
Should I retrain the model when I see drift?
Not automatically. First find the cause. Many fixes are cheaper: update documents, adjust a prompt, correct a data pipeline. Retrain or fine-tune only when tests show the model itself is the gap.
What is the difference between drift monitoring and evaluation?
Monitoring watches live traffic for change, such as new topics, longer outputs or more refusals. Evaluation grades answers against expected results. Monitoring tells you when to look, and evaluation tells you whether quality actually fell, so you need both.
Who should own drift monitoring?
The team that owns the AI feature, with support from engineers who run the pipeline and domain experts who grade samples. Name one owner, because a duty shared by everyone is usually done by no one.
How is drift different from a hallucination?
A hallucination is a single made-up or wrong answer. Drift is a trend, a gradual decline in quality over time. Drift can increase how often hallucinations occur.