Enter your experiment details
Experiment type, sample size, observed lift, run duration, and the primary metric are each scored.
EXPERIMENT SCORING
Experiment Pilot
Enter your experiment's setup and early results to get a calculated confidence score and a clear ship / kill / keep-running call.
- Uses your inputs in a transparent calculation
- Instant result with practical next steps
- No signup required
Planning guidance only. Validate important decisions with customer evidence and your delivery team.
YOUR INPUTS
Build your result
Complete every field. The result updates only when you choose Calculate.
How it works
We calculate a confidence score
Sample adequacy, run duration, and lift magnitude are weighted into a 0-100 confidence score.
Get a clear call
Based on confidence and lift direction, you get a ship, kill, keep-running, or too-early verdict with next steps.
Continue learning: Measuring product-market fit at MVP stage · MVP UX metrics founders should measure
Frequently asked questions
Is this a statistical significance calculator?
No. It's a fast heuristic combining sample size, run duration, and lift magnitude into a directional confidence score — not a p-value or Bayesian posterior. Use a dedicated significance calculator for a rigorous test.
Why does run duration matter if I already have enough samples?
Short runs are prone to novelty effects and day-of-week bias. The score rewards longer runs even at similar sample sizes to discourage calling a winner too early.
What if my lift is negative?
A negative lift with high confidence returns a 'Kill it' verdict — the variant is underperforming the control with reasonable reliability.
Can I use this for more than A/B tests?
Yes — it works for any two-variant comparison (onboarding flows, pricing, messaging, or feature flags) as long as you can report participant count and lift.
How We Compare
| Feature | MVPHub | Optimizely | PostHog |
|---|---|---|---|
| Quick single-experiment confidence check | Included | Limited | Limited |
| No signup or SDK install required | Included | Not included | Not included |
| Automated experiment tracking across your app | Not included | Included | Included |
| Statistical significance / Bayesian testing engine | Not included | Included | Included |
Optimizely and PostHog run and track experiments automatically across your live product once integrated. MVPHub gives a fast, manual confidence check for a single experiment using numbers you already have.
Related tools
Embed this tool
Add this tool to your site with the canonical iframe below. It remains hosted and maintained by MVPHub.