Alumni notes
These are edited with permission. We keep the odd complaint because conversion lift measurement is slow work and pretending otherwise would be a marketing holdout of our own.
Holdout Architecture for Paid Media was the first time someone walked our retailer-network leakage with a red pen. We cut a prospecting line that had been “working” on last click for a year. I still think the three-week pace is tight if your logs are a swamp.
Amelia Brooks · paid media lead, homeware
Rated 8.4 in our internal LMS after Lift Studio. The geo week earned most of that. The SQL examples were thinner than I wanted.
Platform-style note · cohort 24
Helen marked my finance memo as “polite and empty.” Fair. I rewrote it without the adjectives and the CFO asked better questions. I would not send a junior on this course alone; you need scars from a real reporting cycle.
James · Bristol
We used the In-house Measurement Desk because three directors could not agree what “incremental checkout” meant. The shared lexicon stuck. The test we designed still has not launched because legal is nervous about untreated postcodes. That is not Techvectorbase’s fault, but it is the honest ending.
Client in grocery retail
Case note
A brand that already ran MMM asked Lift Studio alumni on staff to design a six-week geo test around a spring burst. Matched markets kept collapsing because of a festival weekend and a storm in the North West. The studio method — treat weather as a covariate, drop the week rather than “adjust it in the narrative” — produced a lift estimate smaller than the agency’s last-click story and large enough for finance to keep a reduced burst the following year.
Case note
Reading Lift in Product Analytics alumni redesigned an onboarding experiment that had been randomising users while a family plan shared a household. Interference was obvious once named. They moved the unit to household and accepted a longer runtime. Conversion lift on the paywall was no longer significant; they shipped a smaller change anyway because support tickets fell. The course did not tell them to worship the p-value.