summarise first or send the whole 200 page pdf, $600 a month ceiling token-cost

Do the arithmetic first, because it eliminates one option outright. 400 questions a day at 120k input tokens is 48 million tokens a day. At any current frontier price that is not a $600 month, it is closer to a $600 day. Full context per question is out unless the context is cached.

What is left:

• Cache the document and let a session ask many questions against it. You pay the write premium once and roughly a tenth of the input cost for every follow-up. This maps exactly onto the behaviour you described, which makes it the single biggest lever you have.
• Retrieval, top 8 chunks, call it 6k tokens a question. 2.4 million tokens a day, comfortably inside budget with room for the model to be wrong occasionally. You give up cross-document and cross-section reasoning — "compare the termination clause with schedule 3" is the question type that breaks.
• Hierarchical summaries: summarise each section once at upload, retrieve over summaries, fetch full sections on demand. Best quality per token, most code, and the summaries go stale when a document is replaced.

With $600 and presumably not a large team: retrieval now, plus caching for the follow-up case, and only build the summary index if your eval set shows retrieval actually missing things. Do not build it because it sounds better.

Chunk on structure, not on a character count. Long documents have headings, numbered clauses, tables — split on those, and put the heading path into the chunk text so retrieval has something semantic to grab and the model knows where the fragment came from.

Most of long-document RAG's bad reputation comes from splitting every 800 characters and cutting sentences in half.

Use hybrid retrieval rather than pure vectors. BM25 alongside embeddings, fused. Document QA is full of exact tokens — defined terms, section numbers, party names, dates — and embeddings are consistently mediocre at exact matching. This is usually a bigger quality win than any reranker and it costs you a few milliseconds.

Route by question type before you spend anything else. A large fraction of what users call questions are lookups — find this term, what date, who signed. Those can go to a cheap model with tight retrieval and be correct. Reserve the expensive path for questions that require reading across sections. Two tiers is usually a 40-60% saving with no visible quality cost.

Check what you can do once at upload rather than per question. Structure extraction, section splitting, summaries, an outline — all of it is per-document work that a lot of systems redo on every single request without anyone noticing.

Whatever you pick, put a hard per-user daily token counter in from day one and return a real message when it trips. Your $600 ceiling is not enforced by your architecture, it is enforced by a counter you have not written yet, and the day someone scripts against your API you will find that out.