Holdout groups are not a luxury

Media teams talk about holdouts as if they were a philanthropic donation to science. They are not. A holdout is a line on the budget that buys you the right to say whether conversions moved because of the intervention. Without it, conversion lift measurement is a mood.

The usual objection is waste. Eight percent of prospecting “left on the table” sounds reckless in a quarter where the CFO is already sour. The quieter waste is a full year of last-click stories that collapse the first time someone asks what would have happened anyway. We have sat in those meetings in Worminster and in London. The holdout is cheaper once you count the strategy that gets rewritten after a bad audit.

Design matters more than the percentage. A 12% holdout that leaks through CRM and retailer networks is theatre. A 6% holdout that still exists in week six, on a unit you can actually starve, will teach you more. That is why Lift Studio spends a week on leakage maps before anyone opens a power calculator.

If your organisation refuses any untreated group, stop calling the work incrementality. Call it reporting. Honesty here saves the fee for a course you cannot use yet.