An agent project becomes easier to evaluate when you can name the exact piece of work it will finish. “Help with growth” leaves too much undefined. “Investigate a lifecycle conversion alert and return an evidence-backed recommendation” gives you something to inspect.
The operating decision is whether a particular implementation can complete useful work at an acceptable cost and error rate. Choosing the workflow comes first.
GREG ISENBERG proposes five criteria for work that might suit an agent: a repeated trigger, stable inputs, clear tools, a measurable finish line, and judgment during execution. That is a useful starting screen. My recommendation is to add two gates before building: whether the work is economically worthwhile and whether its failures can be contained.
Screen one workflow at a time
Start with a recurring task that already has an owner. Describe it in a sentence: when something happens, someone uses these inputs to make this decision or produce this result.
Apply the five criteria using concrete evidence from the existing process.
| Criterion | What to establish | Reason to pause |
|---|---|---|
| Recurring trigger | The event or schedule that starts the work | Nobody can explain when the system should run |
| Usable inputs | Required fields, sources, freshness, and missing-data handling | Each case requires an undocumented search for context |
| Bounded tools | The specific reads and actions the task needs | The proposed access extends well beyond the task |
| Measurable finish line | What makes a completed case correct and useful | Success means only that an output was generated |
| Case-specific judgment | Which decisions change with the evidence | A stable rule can already determine every step |
Stable inputs do not have to mean identical inputs. A preprocessing step might turn different reports into a consistent structure. But someone must define that structure and decide what happens when it cannot be produced.
Likewise, judgment alone does not establish that an agent is necessary. A classifier, a rules engine with human review, or a single model call might handle the uncertain part. Use the screen to expose requirements, then choose an implementation.
Compare the simplest workable options
Consider campaign quality assurance as an illustrative example.
Checking whether a tracking parameter exists, a link responds, or a required field is empty can usually be expressed as fixed checks. Routing those checks through a model adds variability without an obvious benefit.
Assessing whether an ad’s promise matches the landing page requires more interpretation. A model could draft an assessment for review. That still may not require an agent that selects tools and takes multiple steps.
Investigating a mismatch across the ad, landing page, offer, and audience could justify a more flexible workflow. The next source to inspect may depend on the previous finding. Even then, the useful first version might only gather evidence and recommend a correction.
Write down three alternatives before committing: the current human process, deterministic automation, and a bounded model-assisted workflow. Compare them against the same completion criteria. The label matters less than whether the added flexibility earns its cost.
Add value and failure gates
Passing the five-part screen makes a workflow a candidate. It does not make it a priority.
Estimate the opportunity from actual task frequency and handling time. Then include the work automation leaves behind: reviewing outputs, resolving exceptions, repairing errors, maintaining integrations, and updating evaluation cases.
A task with a long manual handling time can still be a poor investment if every result requires equally long verification. Conversely, a modest saving on frequent, easy-to-check work may be useful.
Evaluate failure severity separately from potential savings. A plausible but incorrect internal summary and an incorrect live campaign change have different consequences. High expected value should not cancel an unresolved failure risk in a combined score.
Before selecting a candidate, answer these questions:
- Can a reviewer trace the recommendation to its inputs?
- Can the workflow stop when evidence is missing or contradictory?
- Can an incorrect action be detected and reversed?
- Does the system need permission to make changes, or only to inspect information?
Favor a first test with observable results, limited permissions, and a clear human owner. These are recommendations for making the test interpretable, not guarantees of reliability.
Write a short execution brief
For an illustrative lifecycle anomaly investigator, the brief could specify:
- Trigger: An existing monitoring rule flags an unexpected conversion change.
- Inputs: The metric definition, reporting window, relevant cohort data, and recent change history.
- Tools: Read-only access to approved reporting and change records.
- Completion: A report separating observations, possible explanations, missing evidence, and a recommended next check.
- Judgment: Select follow-up checks based on what earlier checks reveal.
- Stop conditions: Missing required data, incompatible metric definitions, or a request outside the permitted scope.
- Owner: The person responsible for accepting, correcting, or rejecting the result.
The completion definition is the critical part. A polished report is not enough. A report that correctly identifies insufficient evidence may be more useful than a confident diagnosis.
Run a shadow test against the existing process
Keep the normal process in place while the candidate handles the same incoming cases independently. Prevent its output from influencing the human result if you want a meaningful comparison.
Define correctness and material errors before reviewing outputs. Where feasible, have reviewers assess results without knowing which process produced them. Human work is a baseline to inspect, not an automatic answer key.
Include routine cases, incomplete inputs, and exceptions. Track completion quality, material errors, reviewer interventions, turnaround time, and model and tooling costs. Inspect failures individually; an average accuracy figure can conceal the one error type that makes deployment unacceptable.
Estimate human effort saved by subtracting expected review, correction, exception handling, and allocated maintenance time from baseline handling time. Report that estimate alongside the correct-completion rate and total cases attempted. Counting only successful cases would hide the cost of failures.
Shadow testing itself duplicates effort. It can estimate future savings, but it does not demonstrate realized savings. A subsequent limited rollout is needed to check whether people actually spend less time on the work.
Expand only when the evidence supports it
Choose acceptance criteria before the test, based on the workflow’s consequences. There is no universal accuracy threshold that makes an agent ready.
If the candidate cannot outperform a simple rule-based process on the dimensions that matter, use the simpler process. If it produces useful recommendations but needs review, keep review as part of the operating design and its cost calculation.
If it performs adequately within a narrow scope, expand one responsibility at a time. Changing data sources, decision types, or action permissions changes what needs evaluation.
Finally, keep operational improvement separate from growth impact. Faster investigations may free capacity without changing revenue. Claiming incremental growth requires evidence about the downstream decisions and outcomes, beyond a successful workflow test.
Evidence and limitations
The five-part screening framework comes from an X post attributed to GREG ISENBERG. The available text was recovered through a public metadata fallback because official embed metadata was incomplete. An associated image was not analyzed.
The source presents a conceptual framework without comparative results, reliability measurements, cost analysis, or validated thresholds. The examples, execution brief, economic gates, and testing process here are proposed applications. They are not reports of completed experiments or demonstrated business outcomes.
Source basis
- GREG ISENBERG’s public X post proposing five criteria for selecting agent workflows; available text recovered through public metadata.
- Operator interpretation extending the framework with implementation alternatives, economics, failure containment, and prospective evaluation.