Sample size without the theatre

Students arrive with a number they found online: thousands of conversions, a default 80% power, a lift they wish were true. Then the burst is two weeks long because the brand booked the burst. Conversion lift measurement has to start from that constraint, not from a textbook sample that will never be funded.

We size toward a minimum detectable lift the organisation would still act on. If finance would not change spend for a 0.4 percentage point move, do not run a test powered only for that move and then celebrate a “directional” result. Directional is how last-click reporting re-enters the room wearing a lab coat.

Waiting is a cost. So is a noisy two-week window. Write both costs in the design. Sometimes the honest recommendation is not to test this burst at all, and to keep the holdout for a longer always-on line. That sentence has ended more arguments in our studio than any formula.

Bring baseline conversion rates you trust for a quiet month, not a sale week. If you do not trust the baseline, the sample size is decoration. Fix logging first; Lift Studio will wait.