A channel dashboard can tell you which campaign looks efficient this morning. It may leave the harder question unanswered: would moving money into that campaign create more value for the business?
That gap is where measurement needs an operating system. Someone must connect the evidence to a decision, define how much uncertainty is acceptable, and check whether the resulting action worked.
Joe Wyer describes a five-layer measurement stack in a promotional post about Haus: attribution, experiments, response-curve modeling, causally adjusted reporting, and recommendations with verification. The useful idea is that these methods can compensate for one another’s limitations. The post does not establish that the combined stack improves business results.
Here is a proposed way to apply that design principle to one recurring budget decision.
Start with the decision contract
Choose a decision that matters enough to investigate and is narrow enough to evaluate. For example: should the next available acquisition budget go to an existing paid channel, or should spending stay where it is?
Before choosing a measurement tool, write down:
- Action: What can change, and who owns the decision?
- Objective: Which business outcome should improve, over what period?
- Constraints: What limits the change, including capacity and acceptable customer-acquisition cost?
- Evidence: What would justify increasing, holding, or reducing spend?
- Verification: How and when will the team evaluate the action?
A useful objective might be incremental contribution margin over a defined period. Specify whether that margin includes advertising costs. If it excludes them, compare the additional margin with the additional spend before calling the change profitable.
The contract prevents a familiar problem: choosing whichever metric makes the latest recommendation look strongest.
Give each measurement layer a job
1. Attribution supplies operational signals. Use timely reporting to investigate delivery changes, conversion shifts, and differences among campaigns. An attributed conversion count describes credit under a particular rule. It does not, by itself, establish how many conversions advertising caused.
2. Experiments provide causal anchors. Where feasible, use a suitable holdout or randomized design to estimate the effect of a spending policy. Document the population, treatment, comparison, outcome window, and uncertainty. A result from one audience or market has a scope; preserve it when interpreting the finding.
3. Response models support planning. Budget allocation requires an estimate of what additional spending might produce. A response curve can express that relationship, but one experiment at one spending level cannot reveal the entire curve. Additional observations and assumptions are necessary. Make extrapolation beyond tested conditions visible.
4. Calibrated reporting connects slower evidence to faster operations. An experiment may suggest that attributed performance overstates incremental performance under tested conditions. A provisional adjustment can help operators interpret daily reports. It should carry the experiment’s date, scope, and uncertainty. Applying the adjustment more frequently does not create new causal evidence.
5. Verification tests the resulting decision. Record what the system recommended and what the operator actually changed. Then evaluate whether the allocation policy improved the chosen business outcome. A recommendation that agrees with its own model has passed a consistency check, not an outcome test.
These are responsibilities to cover. A team can begin with a decision log, existing reporting, and a feasible experiment rather than buying every component at once.
Keep the handoffs explicit
Most of the judgment sits between the layers.
Suppose an experiment estimates channel impact in a particular market. Before using that estimate elsewhere, ask whether the audience, offer, campaign mix, spending level, and conversion window are sufficiently similar. If they differ materially, treat the estimate as a hypothesis for the new setting.
The same discipline applies when moving from average to marginal returns. Evidence that current spending is productive does not establish that the next budget increase will be equally productive. Allocation decisions need an estimate of the return on the proposed change.
When that estimate is uncertain, reduce the size of the decision or collect more evidence. A precise-looking model output is a poor reason to make a large move that the underlying evidence cannot distinguish from a loss.
Build a recommendation record
For every material change, keep a short record that another operator can understand:
- The proposed budget change and the alternative being considered.
- The expected business outcome, including a plausible range.
- The observations, experiments, and assumptions supporting it.
- The conditions that would make the estimate unreliable.
- The operator’s action and reason for accepting or overriding it.
- The verification design and decision date.
Separate forecasts from observations. “The model predicts improvement” and “the test measured improvement” should never occupy the same field.
Keep rejected recommendations too. They can reveal missing constraints, such as delivery capacity or a promotion ending, that the recommendation process needs to incorporate. Acceptance rate alone says little about recommendation quality.
Test the allocation policy
The next question is whether this process produces better decisions than the current process.
Where feasible, compare the existing allocation policy with an experiment-calibrated policy across comparable units assigned at random. Keep total spending comparable, specify adjustment rules before launch, and measure outcomes over a window that fits the business.
That comparison estimates the difference between the two policies. Estimating absolute advertising incrementality may require a separate holdout. Avoid combining overlapping tests without considering interference: customers, auctions, or campaigns can cross the boundaries a design assumes are separate.
Track the primary economic outcome alongside constraints such as customer volume, retention, operational capacity, and allocation volatility. Define how conflicting results will affect the decision before seeing them.
If the business cannot support a credible comparison, make bounded changes and document what remains uncertain. A before-and-after improvement can inform the next investigation without proving the policy caused it.
Watch for predictable failures
Permanent calibration. An old adjustment becomes a fixture while the audience and campaign mix change. Give calibrations review dates and conditions that trigger retesting.
Uncertainty disappearing between layers. A noisy experimental estimate enters a model and emerges as an exact budget recommendation. Preserve ranges and show whether the decision changes under plausible assumptions.
Verification using the original attribution metric alone. A policy designed to improve incremental outcomes is judged only by attributed ROAS. Return to the decision contract and measure the outcome it specified.
Complexity exceeding decision value. Maintaining the system costs time and attention. Start with one material allocation decision and expand only when the additional evidence can change an action worth taking.
Evidence and limitations
This framework draws on Joe Wyer’s public post describing a five-layer measurement stack associated with Haus. The available source is a concise promotional account. It provides no controlled comparisons, effect estimates, customer results, or detailed model-validation methods.
The decision contract, recommendation record, and implementation sequence above are operator recommendations developed from that architecture. They are not a documented deployment or evidence of improved performance.
Experiments can be limited by low power, interference, measurement error, and weak transfer to other settings. Modeling and calibration introduce further assumptions. Treat this system as a proposed way to make those assumptions visible and test decisions—not as a guarantee that combining measurement methods produces growth.
Source basis
- Joe Wyer’s public post describing a five-layer marketing measurement stack associated with Haus.
- Original operating recommendations synthesized from the stack’s design principle; no independent performance evidence was supplied.