Enter your test's sample sizes and results
Control and variant sample sizes, conversion rates, test duration, and any unusual traffic are each entered.
A/B TEST VALIDITY CHECK
Variant Guard
Enter your A/B test's sample sizes and results and get a calculated validity score flagging whether the result is safe to trust.
- Uses your inputs in a transparent calculation
- Instant result with practical next steps
- No signup required
Planning guidance only. Validate important decisions with customer evidence and your delivery team.
YOUR INPUTS
Build your result
Complete every field. The result updates only when you choose Calculate.
How it works
We calculate statistical validity
A two-proportion z-test checks significance at 95% confidence, while sample size, traffic balance, test duration, and traffic anomalies are checked against common validity rules.
Get a trust score and specific risk factors
The result lists exactly which risk factors were found — small sample, imbalanced split, short duration, unusual traffic, or lack of significance — so you know what to fix before trusting the result.
Continue learning: How conversion rates support MVP market validation · Which MVP metrics matter?
Frequently asked questions
What statistical test does Variant Guard use?
A simplified two-proportion z-test comparing control and variant conversion rates, checked against a 95% confidence threshold (z ≈ 1.96). It's a genuine, if simplified, statistical computation from your actual sample sizes and conversion rates — not a hardcoded verdict.
Why does test duration matter if the result is already significant?
A test run for under a week can be skewed by day-of-week effects (e.g. weekday vs. weekend behavior) even if it reaches significance early. Variant Guard flags short-duration tests as a risk factor regardless of the z-score.
What counts as 'unusual traffic'?
Anything that could bias the sample independent of the variant itself — a traffic spike from a press mention, a partial outage affecting one group more than the other, or a marketing campaign that changed the visitor mix mid-test.
Is this a replacement for a dedicated A/B testing platform's statistics engine?
No. It's a fast, manual sanity check using numbers you already have. A dedicated experimentation platform will run more rigorous sequential testing and correct for multiple comparisons automatically.
How We Compare
| Feature | MVPHub | Optimizely | VWO |
|---|---|---|---|
| Quick single-test validity check | Included | Limited | Limited |
| Named risk factors (sample size, imbalance, duration, traffic) | Included | Limited | Limited |
| Automated experiment running and traffic splitting | Not included | Included | Included |
| Sequential testing and multi-variant support | Not included | Included | Included |
Optimizely and VWO run and split experiments automatically with more rigorous sequential statistics once integrated. MVPHub gives a fast, manual validity check for a test you've already run, using the numbers you already have.
Related tools
Embed this tool
Add this tool to your site with the canonical iframe below. It remains hosted and maintained by MVPHub.