The source documents change every week and the index does not — how do people actually keep retrieval fresh?

Expanding on deletion, because it has a failure mode that survives even a correct implementation.

If your index and your source can ever disagree, you will eventually retrieve something that no longer exists — a document that was deleted while your job was halfway through, a rename that looked like a delete plus an add, a chunk that failed to delete because of an error nobody looked at. And a stale chunk is worse than a missing one, because the model will use it confidently.

Two defences:

Filter at query time, not only at index time. Store the document id on the chunk and check it against the current set of live documents before you put anything in the prompt. It is cheap and it means the worst case is a missing answer rather than a wrong one.

Make the model cite, and make the citation resolvable. If every claim carries a document id and a section, a stale citation becomes visible — someone clicks it and finds nothing, or finds text that does not match. Without citations, your exact bug is undetectable by anyone who does not already know the answer, which is the situation you are in now.

One more thing worth doing this week regardless of the redesign: run a reconciliation once. Count documents in the source, count distinct document ids in the index, and list the ones in the index that no longer exist. That number tells you how bad the current drift is, and it is usually much larger than people expect.

You are not missing a technique, you are missing a key. Almost everything painful about updates comes from chunks having no stable identity, so the system cannot tell a changed chunk from a new one.

What to store alongside every chunk:

• A stable document id that survives renaming. Not the filename — filenames are the thing that changed when the document was replaced under a new name.
• A chunk index or heading path within the document.
• A content hash of the chunk text.
• A version or updated-at timestamp.

With those, the update becomes cheap and obvious. For each document, chunk it, hash each chunk, compare against what you have:

• Same hash: skip. No embedding call, no write. On a document where one paragraph changed, this skips almost everything, which is where the cost saving comes from.
• Changed or new hash: re-embed just that chunk.
• Present in the index but not in the new document: delete it.

That last one is where people lose. Adding is easy and everyone gets it right; deleting is the hard part and it is exactly your bug — a corrected paragraph is still sitting there being retrieved, because nothing ever removed it.

Do this and a nightly run over twelve thousand documents costs almost nothing on a normal day, because almost nothing changed.

The boring option deserves a defence: rebuild the whole index on a schedule into a new collection, then swap.

The arguments against it are cost and elegance. The arguments for it are that it has no state to get wrong, deletions are handled for free because they simply do not appear in the new build, and a bug in your incremental logic cannot accumulate over months — which is the most common way these systems rot.

At twelve thousand documents this is genuinely affordable, especially if you cache embeddings by content hash. That combination is the sweet spot most people miss: full rebuild, incremental cost. You rebuild the index structure completely, and the hash cache means you only pay to embed the chunks whose text actually changed. You get the simplicity of a rebuild and the cost of an incremental update.

Building into a new collection and swapping also gives you two things that are hard otherwise: the old index stays queryable while the new one builds, so there is no window where retrieval is broken, and if the new build is bad you swap back.

I would start here and only move to true incremental updates when the volume makes it necessary, rather than the other way round. Incremental is where the subtle bugs live, and yours is a subtle bug that ran for two months.