Log a baseline period and later periods
Enter (or edit the pre-filled sample) response-quality metrics for a baseline period plus several later periods: average response length, refusal rate, error rate, and average user rating.
AI ENGINEERING & SECURITY MVP
Model Drift Monitor
Log a few periods of response-quality metrics — response length, refusal rate, error rate, user rating — and get a transparent drift score against your baseline period, with the specific metrics and periods that changed enough to flag.
- Weighted, threshold-based drift scoring
- Flags exactly which period and metric drifted
- Loaded with 5 sample weeks — edit any value
| Period | Avg. length (chars) | Refusal rate (%) | Error rate (%) | User rating (1–5) | Remove |
|---|---|---|---|---|---|
The first row is always the baseline every later period is compared against.
Click Analyze drift to see the drift score, status, and flagged periods.
How it works
Each later period is compared to the baseline
Every metric is converted to a percent change from the baseline value, then weighted by how much it typically signals real quality drift — error rate and refusal rate carry the most weight, response length the least.
Threshold rules flag meaningful drift
A metric is flagged when its change crosses a threshold in the direction that matters (e.g. refusal rate up, user rating down). The composite drift score and per-period flags show exactly where and why quality moved.
Continue learning: How AI coding changes MVP scope decisions · How to manage and version prompts in an AI product
Frequently asked questions
What counts as the baseline?
The first row in the table. Every later period’s metrics are compared against it as percent change, so the baseline should represent a period you consider healthy or "known good".
How is the drift score calculated?
Each metric’s absolute percent change from baseline is weighted (error rate 35%, refusal rate 30%, user rating 20%, response length 15%) and summed, capped at 100. It is a relative signal, not an absolute quality score.
Why are some metrics only flagged in one direction?
A rise in refusal rate or error rate is bad; a rise in user rating is good. The logic only flags the direction that actually indicates degradation for each metric, so an improving metric never triggers a false flag.
Can I use my own logged data instead of the sample?
Yes. Edit any cell, add or remove periods with the table controls, and click Analyze drift again — all computation happens in your browser using the values currently in the table.
Does this connect to my model provider or logging pipeline?
No. This is a client-side calculator for metrics you already have (from your own logs, evals, or a spreadsheet export) — nothing is sent anywhere, and there is no live integration.
What thresholds are used to flag drift?
Response length ±25%, refusal rate and error rate ±20% relative change, and user rating ±10% — chosen as reasonable starting points for a first-pass monitor, not a certified statistical standard.
How We Compare
| Feature | MVPHub | Manual spreadsheet tracking | General-purpose AI assistant |
|---|---|---|---|
| Weighted, threshold-based drift scoring | Included | Limited | Not included |
| Flags the specific period and metric that drifted | Included | Limited | Limited |
| Transparent, inspectable calculation | Included | Included | Not included |
| Runs instantly with no setup | Included | Included | Included |
A spreadsheet can track the same raw numbers but leaves the "is this meaningful drift" judgment to you; a general-purpose assistant can reason about pasted numbers but won’t apply a consistent weighted rule every time. MVPHub applies the same transparent threshold logic to every period.
Related tools
Embed this tool
Add this tool to your site with the canonical iframe below. It remains hosted and maintained by MVPHub.