You are not missing a technique, you are missing a key. Almost everything painful about updates comes from chunks having no stable identity, so the system cannot tell a changed chunk from a new one.
What to store alongside every chunk:
- A stable document id that survives renaming. Not the filename — filenames are the thing that changed when the document was replaced under a new name.
- A chunk index or heading path within the document.
- A content hash of the chunk text.
- A version or updated-at timestamp.
With those, the update becomes cheap and obvious. For each document, chunk it, hash each chunk, compare against what you have:
- Same hash: skip. No embedding call, no write. On a document where one paragraph changed, this skips almost everything, which is where the cost saving comes from.
- Changed or new hash: re-embed just that chunk.
- Present in the index but not in the new document: delete it.
That last one is where people lose. Adding is easy and everyone gets it right; deleting is the hard part and it is exactly your bug — a corrected paragraph is still sitting there being retrieved, because nothing ever removed it.
Do this and a nightly run over twelve thousand documents costs almost nothing on a normal day, because almost nothing changed.