An AI that remembers can be lied to once and keep the lie. A filter will not catch it.

AI assistants used to forget everything when the chat closed. The newer ones remember: your prices, your clients' names, the correction you made last Tuesday. That memory is the reason to use them, and it changes the risk. A study published in August puts the change in one sentence: "Agents that remember are agents that can be lied to durably."[1] It then measures what that costs, using the plainest lie it could build: ordinary sentences stating wrong facts, with no hidden instruction in them. A small dose did large damage, and a filter that is good at its own job let all of it through.

One bad request, or a lie that stays.

Agent memory poisoning means writing false information into an AI assistant's long-term memory so that it comes back as trusted context in later conversations. An assistant without memory can be fooled, but only for the length of one conversation. Whatever goes wrong ends when the chat ends. An assistant with memory is different: in the paper's words, "a false statement accepted once is retrieved into every future session that matches it."[1] Ask about that client, that price or that policy next month, and the stored statement comes back as context, looking exactly like everything else the assistant knows.

Most of the security work on AI agents so far has gone to a different threat: instructions hidden in content, the attack called prompt injection, where a web page or a document tells the agent to do something its owner never asked for. The defenses are built to spot those instructions. This study is about a threat those defenses, as the author puts it, are not "built to see, because it does not involve instructions at all."[1] The false statement asks the agent to do nothing; it only has to be believed.

1.2% of the memory, two-thirds of its value.

The test bed is LongMemEval, a public benchmark of long chat histories with questions about what was said in them. The author used 120 of its questions, each with its own history of roughly 250 stored memories, one per round of conversation.[1] For each question, a model was given the true answer and asked to write a plausible wrong one, plus three short chat exchanges asserting it, worded like the question so they would compete with the real evidence when the memory was searched. That produced 360 false memories, about 1.2% of the total. No instruction, no trickery aimed at the search engine, one pass of generation with no retries. The paper calls it "the least sophisticated attack in this class."[1]

Accuracy fell from 0.850 to 0.300: the assistant went from answering 85% of the questions correctly to 30%. "The memory retains 35% of its value," the paper says. "Two-thirds of what an uncompromised memory was worth is gone."[1] The search columns explain why. A false memory ranked first for every one of the 120 questions, and false statements made up a fifth of what the assistant was shown. The author credits the surviving 35% to the reading model's tendency to decline to answer, "not to any defense."[1]

Three memories per question, in a history of about 250, was enough. The paper's own reading is that this attack is the floor of its class, not the ceiling: an attacker who tuned the wording against the search would do better.[1]

Why a filter cannot catch it.

The memory system did not take writes on faith. Every new memory passed through a four-stage screen before it was stored: input checks, a detector for passwords and other secrets, rules for injection patterns, and a language-model classifier. On 250 indirect prompt-injection attacks (the kind hidden in content the agent reads) drawn from InjecAgent, a public benchmark, that screen caught 83% of them (0.832 recall).[1] The paper spends a full section showing it is not a straw man.

Against the false statements it refused 0 of 360. "Not a low rate. Zero, on every one."[1] The author argues no better filter of the same kind would change that, and the reason is simple enough to state in one line: "Content screening addresses payloads; this attack has no payload."[1] An injection carries something a filter can look for, an instruction, a command, a request to send data somewhere. A false fact carries nothing. Take a sentence like "the client moved their renewal to March" (our example, not the paper's). It reads the same whether it is true or false. Only something outside the sentence can settle it: the contract, the calendar, the client. The paper's phrasing: telling a false assertion from a true one "generally requires external grounding beyond the text being screened."[1]

None of this makes the filter useless. The paper is clear that on the attacks it was designed for, it is strong and cheap. "Both statements are true, and the second is not a criticism of the first. They describe a boundary."[1] A filter belongs on the write path. It is not, on its own, a memory defense.

The trap in down-weighting.

If the write path cannot see the lie, the read path has to carry the defense. The system's answer was to rank memories by how closely they match the question, plus a bonus for coming through a trusted channel, so that material from an untrusted channel is preferred less. In the test, the false memories were written as arriving through that untrusted channel. The author stresses that the label marks the channel, not whether the content is malicious: the system has no way of knowing which memories are false.[1] The baseline and three results, from the paper's tables:

The light setting failed for an arithmetic reason the paper derives in three lines: the trust bonus could outbid similarity only within a margin of 0.175, and false statements worded like the question gained 0.32 in similarity over the genuine evidence.[1] The heavy setting worked, then the author checked why. Its penalty on untrusted material was larger than half of the whole range similarity could contribute, so untrusted material was not ranked lower, it was effectively never ranked. When the real answers were the untrusted ones, not one of them was retrieved for any of the 120 questions.

The paper calls the heavy setting "a denial-of-service primitive against the memory" (in plain terms, a way to make the memory useless): anyone who can route genuine content through an untrusted channel, "an adversary, or merely an integration that labels a legitimate source conservatively," makes it permanently unretrievable.[1] It fails gracefully, at least: 96.2% of answers in that run were explicit abstentions (the assistant said it did not know) rather than made-up answers.

The finding is about one design, adding a trust score to a similarity score, under the similarity spread the author measured: "In this measured similarity regime, we find no usable setting for the additive provenance term."[1] It is not a finding that provenance does not work. The author proposes a different shape, a cap on how much of what the assistant is shown can come from untrusted channels, reserving room instead of subtracting points, and is plain that it was "motivated" but not built or tested.[1] The paper adds that a cap still depends on the trust label being right, and that a cap and write-time origin binding (fixing each memory's trust label when it is written, so it cannot later be passed off as trusted) "are complementary, and only the pair is a defense."[1]

For a buyer, the practical question is how a tool applies trust, not whether it has a trust setting. If a tool adds a trust bonus to a similarity score, the paper's arithmetic applies to it; how hard it bites depends on how spread out that tool's similarity scores are, and the paper tested no other system.

Three lines of memory hygiene.

You do not need a security product for this, only a few habits about what your assistant is allowed to remember and how you check it. Nor does it need an attacker. A wrong figure in a pasted document, stored once, comes back in every conversation that touches it. That last point is our reading, not a result of the paper, but the path through memory is the same one. These three habits are our recommendation, built on the paper's argument; the paper does not propose them in this form.

1. Control what can write.

   List every channel that can put something into your assistant's memory: your own conversations, pasted documents, forwarded email, pages it reads, other tools connected to it. Let only the channels you trust write lasting facts. For the rest, either no memory writes at all, or writes that wait for a person to confirm. Facts that move money or commitments (prices, policies, client details, deadlines) get checked against their source before they are stored. That check is the external grounding the paper says a filter cannot supply.

   → a short written list: who and what may write to memory
2. Record where every memory came from.

   Source, channel, date, and who confirmed it. The author logged the identifier of every injected memory as it was written, so the poisoned store could be restored to its exact clean state. The same habit lets you remove everything one bad document wrote in a single pass. Without it, a wrong fact is found one wrong answer at a time.

3. Audit the store on a cadence.

   Once a month, read the memories behind the facts that matter and check them against the source. Keep a fixed set of questions you know the answers to, and rerun it after any change to how memory is written or searched. Count right answers, the paper's "utility retained," not just whether anything looks attacked: a defense that blocks the attack by blocking your own evidence looks like a success on the wrong measure. Our six-line eval suite is a starting shape for that question set.

The benign version of the same problem, a correct fact that later stops being correct, is covered in our note on what happens when you change your mind, and the write, manage and read loop behind all of it in our memory primer.

What this does not claim.

One study, one author. One memory benchmark family, one search setup, one embedding model, one reading model. Nothing here has been independently replicated, and the author says plainly: "We do not know how much of either result is model-specific."[1]

The attacker did not adapt. The false statements were written in one pass and never tuned against the filter or the search. The author calls this the most consequential limitation: "Every attack result here is a lower bound, and every defense result an upper bound."[1]

The no-usable-setting result is narrow. It covers adding a trust score to a similarity score, under the measured similarity spread. It says nothing against provenance designs that reserve space instead, and the author's proposed alternative is untested.

The trust labels were assumed right. The study assumes no one can move a memory into a more trusted channel. The paper cites concurrent work arguing that an untrusted origin can be laundered, for example through the agent's own summaries, and calls that assumption load-bearing.[1]

Chat memory, not business records. LongMemEval is conversational personal-assistant memory. The author does not know how the results transfer to operational memory, and says so.[1]

The test corpora bracket the cost; they do not locate it. The run where all the real answers came through the untrusted channel is a constructed worst case, and the run where untrusted material was mostly harmless filler is an optimistic one: there, the heavy setting lifted right answers from 31.7% to 70.0%, partly because it also removed the filler. The paper makes no claim about where real deployments sit.[1]

Not a verdict on any product. The system attacked is the author's own memory layer, released as open source and tested at its shipped defaults, and the paper reports those defaults failing. Nothing here is a finding about any other vendor's memory feature. No client record is described.

Setting up an assistant that will remember things about your business? Describe the setup through the contact form and you get a free written first read on which channels should be allowed to write to its memory, and how to keep a record of what it stored. Agent work like this falls under our assistants and agents services.