Ask
198
@wonder_why_wren ·

answers get noticeably worse past ~60k tokens even though the window is 200k context-window

Long refactoring session, one conversation, big monorepo. First hour is excellent. Somewhere around the point where the context indicator says 60-70k it starts:

  • reintroducing a bug we fixed 40 minutes earlier
  • referring to a function signature from two versions ago
  • confidently describing a file it hasn't read in this session

Nothing errors. It just gets subtly worse in a way that costs me more time than a hard failure would, because I keep trusting it for another twenty minutes before I notice.

The window is 200k. Why does it degrade at a third of that, and what's the working pattern — new session per task, summarise and restart, something smarter?

12 answers Share
Report

Answering anonymously — a moderator will review it first.

  • @coworking_cass · 3h ago · 3 replies

    Because a long window is a capacity, not a promise about attention. Everything in there competes, and by 60k a large fraction of your context is stale — old file versions, superseded plans, tool output nobody needs any more. The model has no way to know that the version of parseInvoice from turn 6 is dead. Both versions are equally present and equally plausible.

    What works:

    • one task, one session. When the task changes, start over. This is the biggest single one
    • keep the durable stuff in files, not in the conversation. A PLAN.md you edit as you go survives a restart; a plan described in turn 4 doesn't
    • after a significant change, tell it to re-read the file instead of relying on what it saw earlier. Cheap, and it kills the stale-signature class of error outright
    • when you feel the wobble, don't fight it. Ask for a handoff summary — what's done, what's left, which files matter — and paste that into a fresh session

    Restarting feels like losing progress. It costs ninety seconds and it's usually the fastest thing available to you.

    187
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @wonder_why_wren · 3h ago

      'Both versions are equally present and equally plausible' is the clearest description of this I've read. Moved my plan into a file this morning.

      54
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @heirloom_hank · 3h ago

      The handoff summary is underrated. I keep a saved prompt for it — done / remaining / files / gotchas — and it makes restarts close to free.

      33
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @iris_tanaka · 8h ago · 2 replies

    Something you can actually measure: work out how much of that 60k is tool output you'll never need again. In my sessions it's usually 40-60% — whole file reads where I needed one function, test output from runs that passed, directory listings.

    Reading narrower ranges and dropping output you're done with pushes the useful ceiling out a long way. I get roughly twice as far into a task now, changing nothing else.

    79
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @drainfield_dez · 3h ago

      Same experience. The failure isn't 'context is full', it's 'context is 60% garbage', and those two have completely different fixes.

      21
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @muslin_mira · 3h ago

    Worth being precise about the reintroducing-a-fixed-bug case, because it has a specific cause. If the old buggy version of the file is still sitting in context from an earlier read, and it writes from memory rather than re-reading, you get the old version back. It isn't forgetting the fix, it's copying the wrong source.

    Force a re-read before any write to a file that's been edited since it was last read. Some setups do this for you. If yours doesn't, say it explicitly.

    47
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @drainfield_dez · 3h ago

    Costs nothing and helps a lot: commit constantly. Every time it does something you like, commit. Then when it drifts you reset to the last good point and restart the session, instead of trying to talk it back into a state it can't reconstruct.

    I commit maybe 20 times a session and squash later. Cheap checkpoints are what make the restart strategy usable.

    26
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @r2_bucketeer · 3h ago · 3 replies

    just use a model with a 1m window and stop worrying about it

    21
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
    • @coworking_cass · 3h ago

      Bigger windows move the wall, they don't remove it. Same degradation shape, further out. And you will fill 1M with the same 60% garbage and hit it anyway.

      18
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
    • @warranty_wes · 3h ago

      yeah ok. I've seen this happen at 300k too now that I think about it.

      9
      Share
      Reply

      Answering anonymously — a moderator will review it first.

      Report
  • @rubberduck_rae · 3h ago

    Prompt caching interacts with this in a way people don't think about. Staying in one long conversation is cheaper per turn because the prefix is cached, which quietly incentivises you to stay in a degrading session. The saving is real and it is much smaller than the cost of an hour of subtly wrong work.

    22
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report
  • @mature_student_j · 3h ago

    Anecdotally the wobble point moves depending on how homogeneous the context is. Twelve similar TypeScript files degrade earlier for me than four very different ones, presumably because everything starts looking like everything else. No idea if that generalises, but it changed how I batch work.

    10
    Share
    Reply

    Answering anonymously — a moderator will review it first.

    Report