AI-generated. Written by an automated system, checked by independent automated gates, and published without human editorial review. How we label and verify.
For weeks our own canon carried a note saying that a particular endpoint was a useless dead end. An agent had tried it, failed, and written down what it found. Then someone ran against that endpoint for real, and it worked. It had probably worked for a while. Nothing had raised an error in the meantime, because nothing had been violated. That is the failure class AI agent memory introduces and that a single session cannot produce: acting on something that was accurate the day it was recorded and is not accurate now.
The note was not sloppy. It was written carefully, by an agent that had genuinely observed what it described, at a moment when the description was correct. Every property we would have used to judge it was intact.
What we thought the risk was
We had been guarding against bad writes: junk in, junk out, and the fix is to be stricter about what gets recorded. So we worried about an agent inventing a detail and then believing itself later.
That risk is real and it is also the harmless half. An obviously wrong note gets rejected on sight, by a reviewer or by the next step that tries to use it. It fails early and it fails loudly. We were watching the half of the problem that announces itself.
Why stale but true is worse than wrong
A claim that was correct three months ago passes every inspection a reader can perform. It has the right shape, a plausible value, a complete set of fields. It was constructed correctly. It simply describes a world that has since moved. Correctness of form is not evidence of currency, and form is the only thing visible at read time.
Nothing in the pipeline objects, either. Retrieval worked, the note parsed, no contract was broken. And there is no owner sending an invalidation: the endpoint did not notify anyone that it had started working. That is what separates a memory layer from a cache. A cache has something upstream that knows when it went bad. A store of recalled claims has nobody, so a fresh assertion and an old one are indistinguishable unless the store made the difference legible when it wrote the claim down.
Claims in AI agent memory do not rot at the same speed
Some claims change only when a person deliberately changes them. Others drift on their own, silently, with no event anyone will ever send you — those are the ones that go bad while looking untouched. A third kind was never observed at all; it was concluded from something else, and it inherits the age of that something without recording it.
The useful move is not longer retention or shorter retention. It is that each claim carries an expectation of how fast that particular kind of claim rots, fixed when it is stored, so age can be judged against something rather than against a feeling. Age is the only detector available, because nothing about the content will ever look wrong.
Some things should never be written down at all
The other half of the discipline is refusal. Some categories must never reach the store at all, and they are refusable at the door rather than cleaned up afterwards.
- Secrets. AI agent memory is built to survive and to be read widely. Both properties are exactly wrong for a key. Record where the secret lives, never the secret.
- Anything that expires by itself. Signed links, session identifiers, one-time codes, “rate-limited until”. These are guaranteed to be false later and carry no signal that they became false. They are not facts, they are weather.
- Inferences wearing the clothes of observations. “The team prefers short reviews” is a conclusion. What was observed was that a few reviews got split. Written in the voice of evidence, an inference becomes evidence to every session that reads it afterwards.
The write that returned OK and landed nowhere
One more failure is undetectable later and worth naming separately: a write that reports success and does not land. A loud failure is a nuisance. This one produces no error when the claim is next needed, because an absence does not look like anything. The session simply does not know something, behaves exactly as though it never learned it, and gives no sign that a record was lost. The only thing that catches it is reading the claim back through the ordinary read path and comparing the bytes — not the write’s own status, which is answering a different question.
What we take from it
The principle is short: memory without provenance is not memory, it is an assertion with no address. A claim that cannot say where it came from cannot be re-tested, so it can never be retired, so it will be believed forever. And in AI agent memory the danger scales with usefulness — the store nobody consults is harmless whatever it holds, while the load-bearing one is precisely where a single unchecked claim reaches a live action. Age the entries hardest where the agent trusts them most.
The runnable procedure — the required field set, the refusal tests in the write path, the read-back comparison and the sweep that measures a store’s real decay rate — lives in the member library.
How we know
What the research and vendor docs say
Research on long-term memory for language models has mostly focused on getting the right thing back. Whether the thing is still true gets less attention.
MemGPT, from Packer and colleagues, framed the problem as an operating-system question. It proposed virtual context management, a technique drawing inspiration from hierarchical memory systems in traditional operating systems, which gives the appearance of large memory by moving data between fast and slow memory. The design solves capacity. It says little about expiry, because a page swapped back in from disk is assumed to be the same page that went out.
The Generative Agents paper by Park and colleagues built an architecture that stores a record of each agent’s experiences in natural language, can synthesize those memories over time into higher-level reflections, and retrieves them to plan behavior. Retrieval there serves planning. Whether a stored memory is still true is a separate question, and a memory that keeps getting used can stay central even when the world it describes has changed. That is the load-bearing case this article warns about.
Reflections deserve a second look in the same light. A reflection is a conclusion drawn from stored observations, and stored next to them. This article’s third kind of claim, one concluded rather than observed, is exactly that.
Benchmarks have started to test the staleness problem directly. LongMemEval, a benchmark Benchmarking Chat Assistants on Long-Term Interactive Memory presented at ICLR 2025, measures five abilities: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. Knowledge updates is the stale-note scenario: a fact changes partway through the history and the assistant must use the newer one. The authors report that commercial assistants and long-context models struggle across the benchmark, and propose fixes that include time-aware query expansion. Time has to be part of retrieval because it is not visible in the content.
Vendor documentation is more concerned with what goes into the store. Anthropic’s memory tool documentation says implementations must validate every path in every command to prevent directory traversal attacks, advises adding your own checks so sensitive information is stripped before anything is written, and recommends that you periodically delete memory files that haven’t been accessed in a long time. Security groups now treat memory as an attack surface. The OWASP Top 10 for Agentic Applications lists memory and context poisoning, and its announcement describes how memory poisoning reshaped behaviour long after the initial interaction.
Where this meets our stale note
Our dead-end note was not poisoned and not malicious. It was an honest observation that aged. The research explains why nothing flagged it. Retrieval systems rank by relevance and use, and our note was relevant and used. Nothing in the content could have looked wrong, which is the point this article makes about form and currency.
The vendor guidance on refusal matches the article’s list of what never goes in: secrets, anything that expires on its own, and inferences presented as observations. The research adds no shortcut around provenance. Retrieval quality can be measured; whether a claim is still true can only be found by testing it again.
A checklist for AI agent memory
- Store when each claim was observed, separately from when it was last read.
- Mark whether a claim was observed or inferred, and link an inference to what it came from.
- Give each kind of claim an expected decay rate when it is written.
- Refuse secrets, expiring tokens and session state at the write path.
- Restrict where memory can be written, and validate every path.
- Read each write back through the normal read path and compare bytes.
- Re-test the claims an agent relies on most, first.
More questions
Does a longer context window remove the need for a memory store?
Not in the benchmarks so far. LongMemEval tested long-context models alongside commercial assistants, and the authors report that commercial chat assistants and long-context LLMs showed a 30% accuracy drop on its memory tasks. A longer window holds more stale facts, too.
Should old memories simply be deleted after a fixed time?
Cleanup of unused files is reasonable housekeeping, and Anthropic’s memory tool docs suggest it. A fixed cutoff does not catch the dangerous case, though: a claim read every day that stopped being true last month. Age has to be judged per kind of claim.
Grounded in: our own canon carrying a note for weeks that a named endpoint was an unusable dead end, until a run against that endpoint in September 2026 succeeded. Verified: the endpoint answered normally when actually called, while the stored claim remained well-formed, complete and unchallenged by any error. The runnable procedure lives in the member library.