A platform can report strong ROAS while leaving a harder question unanswered: how much business would disappear if you stopped buying those ads?
That question matters most when a channel reaches people who already have a reason to purchase. Retargeting is a useful place to investigate, but low incrementality should be a hypothesis, not the conclusion written before the test.
A post on linkedin.com, attributed in its visible page text to Curtis Howland, recommends connecting holdouts with incremental CPA and contribution margin. The useful operating idea is to make channel reporting answer a budget question. The protocol below develops that idea into a proposed experiment, with explicit economic definitions and rules for inconclusive results.
Start with the decision
Write one sentence before choosing markets or building a dashboard:
“We will decide whether to maintain, reduce, or cautiously increase this channel’s budget based on the incremental contribution it produces over a defined horizon.”
Specify the channel, current spend level, eligible population, and size of the reduction you can actually implement. A complete shutdown and a partial reduction answer different questions.
Choose a primary outcome and a decision date. A useful primary outcome is incremental contribution after media spend per eligible customer, using an eligibility definition fixed before exposure. If that denominator cannot be measured consistently, use a prespecified market-level outcome with appropriate adjustment for market size.
Do not divide by purchasers alone. The treatment can change who purchases, making that denominator part of the outcome.
Define the economics before the experiment
Finance and growth need the same cost boundary. For this protocol:
- Net revenue is revenue after discounts, refunds, and returns, with taxes treated consistently.
- Contribution before media is net revenue minus the variable costs of serving those sales, such as product cost, fulfillment, payment fees, and applicable variable service costs.
- Contribution after media subtracts the tested media spend and any other incremental campaign costs not already included.
Document each included cost and when it becomes observable. If returns arrive after the analysis date, agree on a consistent estimate and a later reconciliation.
The experiment’s economic outcome is the treatment-versus-holdout difference in contribution before media, minus the corresponding difference in media and other incremental campaign costs. Normalize or model the groups consistently before taking those differences.
For acquisition decisions:
Incremental CPA = additional acquisition spend ÷ additional new customers caused by that spend.
Define whether acquisition spend includes media alone or other incremental campaign costs. Keep that definition consistent with the break-even comparison. Orders and new customers are different denominators; a retargeting test covering existing customers cannot silently treat incremental orders as acquisitions.
A break-even acquisition cost is the expected contribution before acquisition expense from an incremental new customer over the chosen horizon. A first-order threshold excludes later purchases. A longer horizon requires defensible repeat-purchase assumptions, uncertainty, and a cash-payback constraint.
The shortcut break-even ROAS = 1 ÷ contribution-margin rate applies when the margin rate is measured before the tested advertising expense, uses the same revenue basis, and captures the relevant variable costs. It also assumes there are no additional acquisition costs outside that calculation. This is contribution break-even, not proof that the business covers fixed costs.
Build a concurrent comparison
Use randomized customer suppression when exposure and identity can be controlled reliably. When that is impractical, consider a geo holdout.
For a geo test, select markets with adequate volume and compare their pre-period outcome levels, trends, promotions, and customer mix. Match comparable markets, then randomly assign within matched pairs to normal spending or suppression.
Keep prospecting, pricing, promotions, and lifecycle treatment stable where possible. Record unavoidable differences. Check that the advertising system can enforce the geographic restriction and that suppressed spend will not simply flow into treatment markets without being accounted for.
Prespecify the analysis. A difference-in-differences estimate compares the change in treatment markets with the change in holdout markets. Similar pre-period trends support the design, but they do not guarantee that the markets will experience identical demand shocks during the test.
Customers crossing boundaries, shared media exposure, and inconsistent location assignment can weaken the contrast. Define how you will detect and assess that contamination before launch.
Let the decision determine duration
“Two weeks” is a calendar choice, not a justification.
Choose duration using baseline variability, available markets, expected effect size, purchase cadence, and conversion lag. Include enough follow-up to observe the outcomes the decision depends on. A first-order test cannot establish long-term customer value simply by ending on schedule.
Before starting, record:
- The smallest economically meaningful effect and whether the design can detect it.
- The exposure period, outcome window, and any follow-up period.
- The primary metric, exclusions, estimator, and uncertainty method.
- Rules for missing data, contamination, and implementation failures.
- Safety stops and the date for the budget decision.
For geo experiments, uncertainty should reflect assignment at the market level. A large number of orders does not compensate automatically for having very few independent markets.
Monitor new customers, first-order revenue, repeat purchases, stock availability, prospecting performance, and assignment or exposure imbalances. Guardrails help identify harm and invalid execution; they do not replace the primary outcome whenever another metric looks more favorable.
Decide with the uncertainty visible
Report the effect estimate and its interval alongside the economic threshold. Agree in advance on the acceptable downside and required return.
Three result patterns need different responses:
- The interval clears the economic hurdle. Maintaining spend is supported under the tested conditions. Any increase should be cautious because the test does not establish returns at a higher budget.
- The interval falls below the hurdle. A reduction is supported, subject to valid execution and the chosen outcome horizon.
- The interval spans meaningful benefit and harm. The result is inconclusive. Follow the pre-agreed risk rule, improve the design, or gather more evidence if doing so is worthwhile.
A statistically unclear revenue change does not establish that revenue was unaffected. If the goal is to show that suppression causes no economically meaningful harm, specify that harm boundary and design the test to evaluate it.
Incremental CPA becomes unstable when estimated incremental acquisitions approach zero. If the estimate is zero, negative, or has uncertainty crossing zero, avoid presenting a precise CPA as a usable efficiency measure. Show spend, acquisition lift, and incremental contribution with their uncertainty instead.
Keep supporting metrics in their proper role
MER—defined here as total net revenue divided by total marketing spend—can monitor overall efficiency. It cannot isolate one channel’s causal effect. State the definition because teams use different revenue and spend boundaries.
Platform attribution can help diagnose delivery and conversion patterns. Post-purchase surveys can reveal discovery paths that click tracking misses. Report survey response counts and coverage; neither source establishes lift on its own.
Avoid turning a survey-to-last-click ratio into a causal multiplier. Likewise, multiplying attributed revenue by a conversion incrementality percentage assumes that incremental and attributed conversions have compatible order values and measurement windows.
If an experiment informs a planning adjustment, give that adjustment a range and a revalidation date. Changes in creative, audience, spend, seasonality, or channel mix can make an old estimate less useful.
Evidence and limitations
The underlying measurement framing comes from a linkedin.com post whose visible page text attributes it to Curtis Howland; structured author metadata was incomplete. The post discusses holdouts, incremental CPA, contribution margin, and business-level efficiency metrics.
Its numerical examples and broad benchmarks lack sufficient supporting data, sample sizes, protocols, and uncertainty estimates to establish reliable expected results. They are not used here as performance evidence.
This article proposes an operating protocol, not a proven case study. A concurrent holdout improves the comparison, but contamination, limited market count, demand shocks, delayed purchases, and cost assumptions can still weaken the conclusion. Any result applies to the population, spend contrast, and time horizon tested; scaling requires another judgment about marginal returns.
Source basis
- A linkedin.com post attributed in visible page text to Curtis Howland on holdouts, incremental CPA, contribution margin, and marketing efficiency.
- A proposed operating protocol that separates causal estimates, economic assumptions, supporting diagnostics, and uncertainty; no validated campaign outcomes are claimed.