Lost in the Middle is an LLM context-retrieval failure mode that causes a model to overlook important information placed near the center of a long prompt. It can produce incorrect answers even when the needed fact is present in the model's context window. The effect is a positional reliability issue, not proof that the model has forgotten or ignored every middle passage.
Lost in the Middle
What is Lost in the Middle in AI?
Lost in the Middle is an LLM context-retrieval failure mode where a model overlooks important information placed near the center of a long prompt. The fact may be inside the context window, yet the model may not use it reliably when forming an answer. This positional bias has been documented in long-context language model research.
Why do LLMs get worse with more context?
Models often perform best when relevant evidence appears near the beginning or end of an input, creating a commonly observed U-shaped accuracy pattern. Long prompts also contain more distractions, repeated facts, and competing instructions. A context window defines how much text the model can accept, not how well it can retrieve every detail. Larger windows can therefore still experience context degradation, sometimes called context rot.
Lost in the Middle vs. text truncation
| Problem | Did evidence enter the context? | Best response |
|---|---|---|
| Lost in the Middle | Yes, but its position reduces use | Curate, reorder, and rerank context |
| Text truncation | No, input exceeded the limit | Shorten or split the input |
| Hallucination | Possibly, but the answer is unsupported | Require evidence and verify output |
| Poor retrieval | Usually no relevant evidence was retrieved | Improve search, chunking, and metadata |
Where the effect causes software failures
The risk is highest when an application sends large, mixed collections of text to an LLM.
- RAG search can miss the policy passage that answers a user question.
- Long support chats can overlook an earlier account detail or constraint.
- Codebase analysis can miss a dependency, security rule, or configuration file.
- Agent tool logs can bury a failed command or API response.
- Requirements documents can cause a generated feature to omit a key acceptance criterion.
- Test generation can skip an edge case described deep in a specification.
How to test for Lost in the Middle
Use a position test rather than trusting a single successful prompt.
- Choose one question with a clear source fact.
- Keep the question and fact unchanged across test cases.
- Move the fact to the beginning, middle, and end of the context.
- Add realistic but irrelevant distractor material.
- Score answer correctness and whether the model cites the right evidence.
How to reduce Lost in the Middle in prompts and RAG
These practices reduce risk, but no prompt layout guarantees reliable retrieval.
- Send only context relevant to the current question.
- Chunk documents into focused, meaningful passages.
- Use query-aware retrieval and reranking before generation.
- Place essential instructions near prompt edges.
- Provide structured summaries for long source collections.
- Ask for evidence citations, then verify them after generation.
Lost in the Middle when building with Emergent
When building an AI feature, keep app requirements and supporting documentation focused, then test outputs against their source material. This is especially important for retrieval-backed search, coding tasks, and automated workflows. Explore related concepts in the Emergent glossary and apply them during app design and development. Build grounded AI apps with Emergent.
Start Building
on Emergent today
on Emergent today
Start Building
