AI's Blindspot: The 'Lost in the Middle' Effect
- 2 hours ago
- 4 min read
There's a weird phenomenon with the way AI reads documents and conversations, and it negatively impacts the accuracy of the responses we get back.
AI's accuracy when recalling information located in the middle of the context window is lower than if the same information were located at the start or the end.
It's known as the 'Lost in the Middle' effect, and it's a problem that researchers have studied in some depth. The consequences of this are pretty simple to understand and can make a real difference to the results we are able to get from AI tools.
A quick note on context
If you haven't come across the term before, a context window is the total amount of text an AI model can hold in its working memory at once. Everything you type and everything you paste in counts toward that limit, along with the model's own replies. Think of it as a whiteboard with a fixed amount of space.
For a full explanation, Context Windows and Tokens covers this in detail.
The important point here is this: fitting inside the context window and being read with equal attention throughout are not the same thing.
What does "Lost in the Middle" mean?
In 2023, researchers published a study titled: "Lost in the Middle - How Language Models Use Long Contexts". They tested how well language models could retrieve specific information from long inputs, systematically varying where in the document that information appeared.
When relevant details appeared near the start of the input, models performed well. Near the end, performance was also reasonably strong. In the middle, there was a notable decrease. This remained true even for models specifically designed to handle long inputs.
Why do AI models favour the start and end of an input?
This comes down to positional bias: the tendency of models to weight the attention they pay to text depending on where it is in the input.
Primacy bias is the preference for information near the start. Recency bias is the preference for information near the end. The model isn't consciously skipping the middle. It's a consequence of how these systems are trained and how attention is distributed across long sequences of text. Either way, details sitting in the middle of a long context are the most at risk of being missed.
The Effect of Filling the Context Window
More recent research (Veseli et al., 2025) adds another layer of complexity. The Lost in the Middle effect isn't constant - it changes depending on how much of the context window your input is occupying.
When your input uses up to around half of the available window, the effect is at its strongest. Primacy bias is high; the start of your input carries disproportionate weight. As the input grows toward the limit of the window, that changes. Primacy bias weakens considerably, and the Recency bias, by contrast, stays stable regardless of input length.
What this means in practice is that the more you push an AI toward its context limit, the less you can rely on the start of your input being treated carefully. However, the accuracy of recalling information at the end of your input remains pretty consistent throughout.
For those curious, the graph of accuracy Vs position in the context looks like this. The different lines show how full the context window is.
How to structure your prompts when working with long documents
Lead with what matters most. For short to moderate-length inputs and conversations, primacy bias works in your favour. Put your key instruction, the specific question you want answered, or the most important context right at the start. Don't bury it after a lengthy preamble.
Repeat critical details near the end. For longer inputs, stating something important near both the start and end gives it a better chance of being registered. It might seem slightly redundant, but it can often be worth it.
Pull out the relevant section. If you're working with a lengthy document and asking about something specific, extract the relevant passage and place it close to your question. Don't assume the model has attended equally to every page. The less you put in the context window, the more accurate the overall response will be.
Break it into chunks. Rather than pasting an entire document at once, asking targeted questions on individual sections is often more reliable than a single pass over everything.
Putting it all together
The way AI reads long documents is not the same as how a person skims one. Position matters. The middle is consistently the most vulnerable part of any long input, and that vulnerability is amplified as the context window fills.
Fortunately, it's quite simple to act on. We can structure our inputs with intent, lead with what matters, and avoid burying critical information where it is least likely to be remembered.
For more on getting better results from AI tools, Getting Started with AI - The Fundamentals of Prompting is a solid starting point. You're Using ChatGPT Wrong - Here's How to Prompt Like a Pro goes further into practical technique.
Comments