A geo-lift test is useful when the answer could change a budget decision. It becomes much less useful when the brief is simply “check whether our attribution is right.”
That question leaves too much open. Which tactic? Which customers? At what spending level? What would you do if the experiment and the dashboard disagreed?
A practitioner post on linkedin.com proposes a useful starting point: isolate one advertising tactic, compare geographic areas with and without it, and use the estimated lift to inform allocation. The operating challenge is making that comparison credible enough to act on.
The playbook below develops that idea into a test brief. It describes a proposed process, not a reported experiment or a proven result.
Start with the decision you can actually make
Write the decision before the hypothesis.
For example: “Should we maintain the selected Meta prospecting tactic in eligible markets at its current spending level?”
That leads to a testable hypothesis: maintaining the tactic generates additional new customers at an acquisition cost below a predefined economic ceiling.
Define that ceiling using the contribution margin and payback assumptions the business accepts. State the customer definition too. An account creation, first order, and first completed order are different outcomes.
Also record what each result would change. Evidence of acceptable incremental economics might support maintaining the tactic. Evidence of unacceptable economics might support reducing it. An imprecise result might leave the decision unresolved.
A holdout at current spend does not establish the return on the next budget increase. If the actual decision is whether to spend more, the experiment needs a spending contrast that addresses that question.
Check feasibility before committing to a holdout
Use historical outcomes from eligible geographic units to assess whether a decision-relevant effect could be detected.
Start with weekly new-customer counts and examine differences in market size, trends, volatility, customer mix, and operating conditions. Similar average sales alone do not establish comparability.
A market with expanding delivery coverage may behave differently from a mature market. Local promotions or stockouts may change acquisition independently of advertising. If those differences dominate the expected effect, a longer test may not solve the design problem.
Have the person responsible for analysis assess power and the minimum detectable effect before launch. The practical question is whether the test can distinguish outcomes that would lead to different budget choices.
Avoid adopting a universal minimum monthly spend or a fixed quarterly testing schedule. Feasibility depends on outcome volume, variation, available markets, the expected effect, and the cost of withholding spend. Test timing should also reflect when a consequential decision is due.
Make the spending contrast unambiguous
For a test of whether to maintain a tactic, a clean proposed design is:
- Select multiple eligible geographic units and group comparable units using historical data.
- Randomly assign units within those groups to continue the tactic or withhold it.
- Maintain the planned spending policy in treatment areas.
- Keep other marketing and promotion policies comparable across groups.
- Record deviations that could affect the comparison.
Withholding spend in controls while simultaneously increasing treatment spend changes the question. It combines the effect of removing the tactic with the effect of increasing its intensity elsewhere.
Budget displacement deserves explicit attention. If money withheld from control areas automatically flows into treatment areas, the intended comparison has changed even if nobody manually raised a budget.
Geographic isolation also has limits. People move, purchase across boundaries, and encounter advertising outside their home area. Define how customers belong to markets and assess whether spillover could materially weaken the contrast.
Freeze the brief before launch
A useful brief should let someone unfamiliar with the project understand both the intervention and the decision.
Include:
- Decision and hypothesis: the budget action being considered and the effect needed to support it.
- Population and assignment: eligible markets, exclusions, matching approach, and randomization.
- Intervention: exactly what continues, what stops, and how budgets will behave.
- Outcome and timing: the new-customer definition, test duration, conversion lag, and observation window.
- Analysis: how historical differences will be adjusted, how uncertainty will be estimated, and how spend and outcomes will be put on a comparable basis.
- Guardrails: tolerable business loss, operational disruptions, and conditions for stopping or invalidating the test.
- Decision rules: what would support maintaining, reducing, or further investigating the tactic.
Monitor delivery availability, stockouts, promotions, spend displacement, and customer quality during the test. Those checks help establish whether the experiment ran as designed.
Avoid repeatedly checking the primary result and ending the test as soon as it looks favorable. Agree on the analysis and stopping approach in advance.
Read incremental acquisition cost carefully
The proposed economic metric is:
Incremental acquisition cost = incremental spend ÷ estimated incremental new customers.
Both parts must represent the same tested contrast. If treatment and control groups differ in size, subtracting their raw totals is not enough. The analysis must account for the design and baseline differences, and express spend and customer effects on a consistent scale.
Report the estimated customer lift and its uncertainty alongside the cost calculation. A single acquisition-cost number can hide a weak estimate.
If estimated lift is near zero, negative, or too uncertain to distinguish useful outcomes, the ratio does not provide a dependable CPA for allocation. Label the result unresolved or unfavorable as appropriate to the evidence; do not force it into a tidy efficiency ranking.
Customer quality matters too. A test focused on first purchases can answer an acquisition question while leaving repeat purchase and longer-term contribution uncertain. Keep those limits visible when applying the economic ceiling.
Reconcile attribution without creating a universal multiplier
Compare the experiment with attributed outcomes over the same markets, dates, customer definition, and conversion window. Otherwise, differences may reflect inconsistent scope before they reveal anything about attribution.
Treat the comparison as a scoped calibration exercise. Record the tactic, markets, spending level, period, uncertainty, and conditions that would make the finding less applicable.
A result for one prospecting tactic does not establish the incrementality of every ad inside it. It also does not establish marginal returns at a higher budget or prove that the same attribution adjustment will hold through a promotion, geographic expansion, or major creative change.
My recommendation is to keep a short calibration register with each experiment’s scope and a reason to revisit it. That gives future budget discussions a usable record without turning a temporary estimate into a permanent assumption.
Evidence and limitations
The underlying idea comes from a practitioner post on linkedin.com recommending geographic holdouts for advertising allocation. The supplied evidence includes no completed experiment, sample size, uncertainty interval, or demonstrated business outcome.
The test brief, decision rules, and calibration register here are operating recommendations developed from that idea and its identified limitations. Claims about universal spending thresholds, testing cadence, or fixed attribution discrepancies are not supported by the available evidence and are not adopted here.
A live test still requires historical data, a feasibility assessment, and an analysis appropriate to its assignment structure. Its conclusions apply to the tactic, markets, period, outcomes, and spending contrast actually tested.
Source basis
- A practitioner post on linkedin.com describing geographic holdouts to inform Meta advertising allocation.
- Critical assessment of the proposed design, incremental acquisition economics, uncertainty, and limits on attribution calibration; no completed experiment results supplied.