Context Rot
Also called: context degradation
Context rot is the observed pattern that a language model's reliability degrades as the amount of text in its input grows — not only when the context window is nearly full, but incrementally as tokens accumulate, even on tasks the model handles easily at short lengths.
A large context window invites a tempting assumption: if the model can accept a million tokens, you can just pour everything in and let it sort out what matters. Context rot is the name — popularized by a 2025 research write-up from Chroma — for the finding that this assumption is wrong. Model performance does not stay flat as input grows; it drifts down, and the drift starts long before the window is full.
The research tested many current models on deliberately simple tasks — find a fact, repeat some text — at a range of input lengths, and found reliability decreasing at each step up in length, not just near the limit. A model with a huge advertised window can still get measurably less dependable at, say, tens of thousands of tokens. The tasks were easy; the length itself was the variable that hurt.
The practical takeaway is the same one context-engineering is built around: more context is not free and not always better. Filling the window with loosely relevant material can make the model worse at using the part that matters. This is why agent harnesses summarize old turns, retrieve narrowly instead of dumping documents, and prune tool output — they are managing context rot, whether or not they call it that.
How it works
The mechanism isn't a single bug; it's a mix of how attention spreads over more tokens, how training data underrepresents very long coherent inputs, and how distracting or near-duplicate content competes with the relevant span. Position matters too — the related "lost in the middle" effect shows material buried mid-context is recalled worse than material at the start or end — but context rot is the broader claim that total length degrades reliability regardless of where the key information sits. Because the decline is gradual, it often goes unnoticed until a task that worked in testing (short prompts) starts failing in production (long, accumulated context).
Example
An agent answers a question correctly when given the 3 relevant paragraphs. Given the same 3 paragraphs plus 40 pages of loosely related documentation pasted in "for completeness," it starts missing details that were right there in the original 3 paragraphs. Nothing about the question got harder; the extra tokens degraded the model's handling of the part that mattered.
How it differs
Context rot vs. lost in the middle: "lost in the middle" is a specific positional effect — facts placed in the middle of a long context are retrieved less reliably than those at the beginning or end. Context rot is the broader observation that overall reliability falls as total input length increases, independent of where the key content is placed.
Common misconceptions
FAQ
What is context rot?
Does a larger context window fix context rot?
How do you deal with context rot in an agent?
Last checked: 2026-08-30