A recurring growth agent needs to remember more than what happened. It needs to know which decisions still apply.
That distinction matters when an agent prepares experiment readouts, compares acquisition cohorts, or recommends the next test. Remembering an old activation definition perfectly can still produce the wrong analysis if the team changed that definition last month.
Persistent memory offers continuity. It also creates another place for outdated assumptions to survive. Before adopting it, test whether the agent makes better decisions with history available.
Separate the record from the summary
VictorTaelin/OptMem documents a useful architecture: raw memories live in an append-only record, while a hierarchy of summaries acts as a rebuildable cache. The agent can retrieve a bounded amount of context without loading its entire history.
The operating appeal is straightforward. Preserve the underlying record, compress it for routine use, and return to the detail when a task requires it.
For growth work, that could mean keeping the original experiment decision while surfacing a short summary during the next planning session. If the summary loses a qualification, the underlying entry remains available for inspection.
But preserving the record does not establish which entry is authoritative. Nor does a summary guarantee that a correction survived compression. Those are behaviors to evaluate.
My recommendation is to treat memory selection as a decision-quality experiment. Storage capacity and retrieval speed matter only after the agent can use the retrieved history correctly.
Start with one recurring task
Choose a narrow workflow with clear answers. An experiment-reporting agent is a useful candidate because its decisions can depend on prior definitions, exclusions, and interpretation rules.
Build a synthetic history with enough variation to expose mistakes. For example:
- An activation event changes, with an explicit effective date.
- A campaign name is corrected after an earlier entry used the wrong label.
- A proposed experiment is discussed but never approved.
- An experiment conclusion applies only to one customer segment.
- A reporting preference changes, while the underlying metric definition stays the same.
These are proposed test scenarios, not observed results. Each should have a written answer key specifying which history applies and why.
Include tasks that require no historical context. Otherwise, the evaluation rewards an agent for retrieving something even when memory should have no influence.
Compare three memory conditions
Run the same task suite under three conditions:
- No persistent memory. Provide the current task and its necessary inputs, without earlier session history.
- A searchable log. Make the historical entries available through a simple retrieval approach.
- Hierarchical summaries. Make the same history available through a system that summarizes and retrieves it in layers.
The first condition shows the cost of missing history. The second tests whether basic retrieval is enough. The third tests whether summarization adds value beyond that simpler option.
Keep the model, task wording, historical entries, and evaluation rules consistent. Set a common context budget, then record how much each condition actually uses. Count any model work needed to create or rebuild summaries separately from the cost of answering tasks.
Reset memory between independent runs so one condition does not inherit answers from another. For tests of learning across sessions, use an identical sequence of updates in each condition. Repeat the tasks enough to see whether a promising answer is dependable rather than a one-off success.
Score application, not recognition
The primary measure should be the share of tasks where the agent correctly applies all relevant prior decisions.
A response that mentions the updated activation definition but calculates the report using the old one fails that test. So does a recommendation that presents an unapproved proposal as settled policy.
Before running the comparison, define what counts as a pass. Give scorers the answer key and hide the memory condition where practical. Record partial successes for diagnosis, while keeping the main pass criterion stable.
Track these failure types separately:
- Missed history: a relevant decision was available but ignored.
- Stale history: an obsolete fact overrode its correction.
- Unsupported history: the agent claimed a prior decision without a supporting record.
- Lost scope: a segment-specific conclusion became a general rule.
- Unnecessary influence: irrelevant history changed an otherwise correct answer.
Ask the agent to identify the record supporting a historical claim. Then verify that the record supports the claim. A plausible reference is insufficient evidence.
Make corrections a first-class test
Appending a correction preserves the sequence of events, but leaves both versions available for retrieval.
For the pilot, give each decision a stable identifier, a recorded date, an applicable period, and a status such as proposed, active, or superseded. Connect corrections to the entries they replace. These are recommended design choices, not capabilities established by the supplied OptMem documentation.
Test both present-day and historical questions. A new definition may govern future reporting while an earlier definition remains necessary to reproduce an old report.
Also test what happens when a summary and a raw entry disagree. Require the agent to inspect the relevant underlying records or acknowledge the unresolved conflict. Quietly choosing whichever version appears first is a failure mode worth catching early.
Set the adoption rule before seeing results
Choose the improvement that would justify additional infrastructure, along with tolerances for errors and cost. There is no universal threshold in the available evidence.
Track prompt tokens, task latency, summary maintenance, setup effort, and the time spent diagnosing wrong answers. A system that saves context but demands frequent manual repairs may be a poor operating tradeoff.
Use a concrete decision rule: adopt hierarchical memory only if it improves correct application over the searchable log enough to justify its additional cost, while staying within your stale-error tolerance. If basic retrieval performs similarly, keep the simpler system.
Limit the pilot to synthetic or deliberately selected non-sensitive material. Recording everything makes later correction, deletion, and access management harder. A rebuildable summary also needs to stop reproducing information removed from its underlying records.
The useful output of this pilot is a failure map: which decisions the agent handles reliably, which require better records, and which still need human judgment.
Evidence and limitations
The architectural starting point comes from VictorTaelin/OptMem’s repository documentation, which describes append-only raw memory and rebuildable hierarchical summaries. That documentation is an author description, not an independent evaluation.
The supplied evidence does not establish improved growth-agent performance, reliable contradiction handling, summary fidelity, or acceptable maintenance cost. Its retrieval-speed claim does not demonstrate those outcomes.
The comparison design, scoring rules, correction structure, and adoption criteria here are recommendations. No experiment results are available. Even a successful synthetic pilot would support only the tested tasks and conditions; performance across longer histories, different models, or shared memory would require further evaluation.
Source basis
- VictorTaelin/OptMem repository documentation describing append-only raw memory and rebuildable hierarchical summaries.
- A proposed synthetic evaluation of recurring growth-analysis tasks, with recall-quality measures and operating-cost guardrails; no measured outcomes.