More Context Isn't Better
Many people think "feed it all the material and it'll be smarter." The opposite is true: stuff it too full and it can't grab the key points.
A Classic Failure: Lost in the Middle
There's a common phenomenon called **"lost in the middle" (Liu et al. 2023, Lost in the Middle): on many models, the model remembers the beginning and end clearly, while the content stuffed in the middle** is most easily overlooked, and the longer the context the more pronounced it gets—though newer long-context models have noticeably eased this.
[开头] 关键指令 ← 记得牢
[中间] 一大堆资料 ← 最容易漏
[结尾] 最新的问题 ← 记得牢Defensive Prompting: Few and Accurate + Restate the Goal
Three practical moves: 1) feed only what's relevant, don't stuff in unrelated material; 2) put key requirements at the start or end; 3) on long tasks, restate the goal every so often.
提醒一下，我们的目标始终是：把价格统一保留两位小数。
现在继续处理下一个文件。自测 · 学完检查一下
想真正动手做题、记进度、攒连胜？到互动课里练。
Judge: the more context you give the model, the better it will surely understand.
答案：False
Stuffing the context too full actually dilutes the key points, making the model more likely to miss things and be led astray.
Why is "don't stuff in unrelated material" a good habit?
答案：Irrelevant info dilutes the key points and raises the chance of drift
Feed only relevant content and the model can more easily lock onto what truly matters.
The model remembers the beginning and end well but most easily overlooks the middle—this phenomenon is called "lost in the ___."
答案：middle
"Lost in the middle" is the classic long-context pit—don't bury key info right in the center.
Judge: putting the most critical requirement right in the middle of a big block of material is the safest position.
答案：False
The exact middle is the most easily overlooked; key requirements should go at the start or end.
Which practice effectively reduces long-context derailment?
答案：Restate the goal every so often during a long task
Periodically restating the goal is like straightening the steering wheel, keeping it from forgetting the task as it goes.
Judge: putting key requirements at the start or end makes them less likely to be missed by the model than burying them in the middle.
答案：True
The model is more sensitive to the start and end; key info placed at these two ends has a higher hit rate.