Equipment manuals, roughly 300 pages each, 40 of them. Numbered sections and subsections, plenty of tables, some two-column pages. Answers need to be precise because people are following procedures. Is heading-based splitting worth the extra parsing code or is fixed-size with overlap good enough?
Fixed 512 with 64 overlap is a perfectly respectable baseline and takes an hour. Heading-aware is better when the headings are reliable, which for machine-generated equipment manuals they usually are and for scanned ones they are not at all. Do the cheap version, measure, then decide whether the parsing code is worth it.
Cheap reliability test before you commit to heading-aware: extract the headings, compare the count against the table of contents in the PDF, and eyeball twenty of them. If they line up you get section paths for free. If your extractor is inventing headings out of bold body text you will find out in five minutes instead of after writing the splitter.
Carry page number and section id in metadata and show them in the answer. For procedures, people trust "section 3.4, page 112" far more than a paragraph with no provenance, and it lets them go and check — which they will, because it is a procedure.
Overlap does not fix a bad boundary. It gives you two chunks that both straddle it.
Handle tables separately. Extract them as units, render as markdown, never let a splitter cut one in half. A chunk containing the bottom third of a torque specification table is worse than no chunk, because it looks authoritative.
@frosting_fixer · 8mo ago · 3 replies
Heading-aware with a size cap: split on headings, then split any section over ~700 tokens on paragraph boundaries so one enormous section does not become one useless chunk.
But the thing that will move your numbers most is smaller than either option — prefix every chunk with its heading path.
It gives the embedding topic context that the paragraph itself often omits ("then turn the valve clockwise" means nothing standalone), and it gives the model a breadcrumb to cite. Costs 15 tokens a chunk.
And before any of this: look at what your PDF extractor produces. Two-column pages and tables are where naive extraction turns text into nonsense, and no chunking strategy recovers from that.
Reply
Report
@layer_shift_lu · 8mo ago
Our extractor was interleaving the two columns line by line. Every chunk from those pages was word salad and I had spent two days tuning overlap on top of it.
Reply
Report
@mise_en_mess · 8mo ago
Print 20 random chunks and read them. That is the entire debugging technique for this stage and almost nobody does it before reaching for a fancier splitter.
Reply
Report