Rigorously

A practice environment for designing rigorous online experiments — power, CUPED variance reduction, common pitfalls.

Problem

• Most "A/B testing tutorials" stop at "alpha = 0.05, sample size formula, done." That's where the actual problems start.
• Most experimentation goes wrong because the design was sloppy, not because the analysis was.

Approach

• A step-through coach that drills the design choices that matter.
• It walks through framing the metric, choosing a pre-period for CUPED, doing power analysis honestly, and the diagnostic moves when your readout looks suspicious.

Outcome

• The habits it drills come from running real experiments on a production conversational-AI system, where sloppy design costs weeks of traffic.
• A rehosted public version is in the works.