You have felt this. You ask the model for one thing, it delivers, and then you say "actually, do X instead," and the quality falls off a step. It is not your imagination, and it is not just your model. In July 2026, Microsoft Research put a number on it across nine models, frontier and open, and the drop showed up in every single one.
Strong single-turn scores do not survive the chat.
The method is the clever part, and it earns one sentence because it is why the numbers are trustworthy in shape. Take a benchmark question with a known answer, pin that question as the last thing the user says, then synthesize a plausible conversation that arrives there: a first request, a correction, a change of plan. The original grader still scores the final answer, so there is no fuzzy "AI judge" in the loop. The paper then runs the same tasks two ways: stated once and fully (they call it single), and delivered as that evolving conversation (evolve).
Here is grade-school math, the easiest domain in the study, where every model answers almost perfectly when you state the problem once.
Same problems, same answers, same models. The only thing that changed is that the goal arrived in pieces, with a correction and a change of direction along the way, and every model lost between 16 and 22 points. Microsoft Research reports the headline as GPT 5.5 sliding from 99.0% to 80.5% after six intent changes. The harder domains (database queries, agentic search, software fixes) drop further, and the ranking does not survive the move: some models that match each other on the static test pull apart once the intent starts shifting.
Adding is cheap. Switching is expensive.
The paper splits how intent moves into exactly three kinds, and they do not cost the same.
Reveal is new detail, same goal. "Find me a restaurant in New York," then "I'm vegan." The picture only grows. Revise is a changed value. New York becomes Brooklyn, and the model has to overwrite one belief. Switch is a changed task. "Find a restaurant" becomes "book it," and the model has to set aside most of what it was doing. Scaled one at a time on the same math, the first two barely register. The third falls off a cliff, and it drags down everything that composes with it.
The authors' explanation is the useful part, and I will attribute it plainly: a task switch "demands a larger belief state update," because the model has to disregard the context it already built rather than absorb more of it. Adding is easy. Invalidating is hard. A follow-up analysis sharpens it: models mostly handle the switch at the moment it happens, then lose ground on whatever comes after it, because they "struggle to jointly integrate the context before the switch with the updates introduced afterwards." The pivot itself is not the problem. Holding the pre-pivot work and the post-pivot correction in one coherent picture, turn after turn, is.
It is not that it forgot.
The obvious fix is to remind the model what you now want. The paper tested exactly that, and the result is why the reminder only half-works.
Two experiments separate two different failures. The first asks the model, at the end of each turn, to state your current intent: can it even track what you want? It nearly can. Tracking holds at 96 to 99% for reveals and revisions, and dips to 82% for task switches (GPT 5.1 on math). So the model is not losing the thread. The second experiment hands the model a perfect restatement of your current goal on every single turn, then measures whether it can act on it. That helps, and it still falls short.
Read the two experiments together. Even when the model knows exactly what you currently want, because you handed it the answer, the old conversation still sitting in its context makes it worse at doing the work. That is context pollution, not amnesia. "Just remind it what you want" is a partial fix by construction, because the superseded conversation is itself the drag, and a reminder adds to the context rather than clearing it. The authors say the other half out loud: a system has to reduce attention to the stale context, not merely restate the current goal.
The same thing surfaces in the software-fix traces, in a coarser form. Under evolving intent the agent pours its tool budget into looking around instead of doing the work: fewer than 4 of every 100 tool calls per turn are actual execution (running the code, editing the file), and the rest are exploration (grep, find, ls). More budget did not help, because the piled-up context and tool history had themselves become distractions.
What this changes about how you work.
This is an unusually applicable paper, because you are the user in the experiment. Every long session with a coding agent or a chat assistant is an evolving-intent conversation: you reveal the goal in pieces, correct values, and sometimes change direction. Four things follow, stated as what the evidence supports.
- A real pivot deserves a fresh session, not a new instruction.
When the task genuinely changes ("actually, let's do X instead"), that is a function switch, the most expensive transition measured, and the damage compounds with every turn after it. Restating the goal and pushing on helps, and does not fully undo it. Starting clean feels wasteful and is the cheapest fix in the study.
target · pivot the task → open a new session - Correcting a value mid-session is fine.
Reveals and revisions cost almost nothing. Adding a constraint or fixing a wrong detail is not what hurts. The expensive move is task-shaped change, not detail-shaped change, so you do not need to restart every time you refine.
target · refine freely · restart only on a real change of task - A drifting agent needs a restart, not a bigger budget.
When a long agentic session starts re-reading and searching instead of editing and testing, that is the documented symptom, not bad luck. More tool budget made it worse in the study. Clear the context and restate the current goal cleanly.
target · drift = clean restart, not more tokens - Discount single-turn scores when you pick a model for collaborative work.
Almost every benchmark number you read is single-turn, and this paper shows the ranking does not survive the drift. A model that wins the static test can lose to one that holds up better under a changing goal. Weight the evolving-intent behavior, not the leaderboard.
target · judge collaborators on drift, not static scores
What we are not claiming.
This is one preprint, and it earns its caveats. There are no error bars and no repeat runs, and the samples are small (50 software tasks, 100 each for database and search, 200 for math), so the direction is the evidence and the exact percentages are not. Three of the software-fix scores are literally 0.0, and they are a scaffold artifact, not a capability result: those models ran out of tool budget and timed out rather than answering wrong, and the budget was raised for two other models but not for them. We kept every one of those cells out of the figures above. The tool that wrote the simulated conversations, GPT 5.1, was itself one of the nine models being graded, which the paper does not address. And the simulated users are scripted and stylistically uniform, not real operators in a real repository. The direction is well supported because it appears in every model and every domain tested. Treat the size of the effect in your own work as unknown, and the shape of it as real.
If you are choosing a model for collaborative work, or your long sessions keep drifting off course, the contact form is the way in. Send us the workflow and we will send back a written read, no call required.