32b at q4_k_m or 14b at q6 on 16gb vram when i need strict json out quantisation

Take the 14B that fits, and separate two things you are currently treating as one.

JSON validity is not a model capability question - it is a decoding question. Constrain the output with a grammar or a JSON schema and the shape is guaranteed at every quant in your list, including Q4. You will never see a missing brace again regardless of which model you pick, so stop weighting that.

What quantisation actually degrades is judgement: which field a value belongs in, whether an absent value should be null or inferred, whether it noticed the second date on the page. That degradation is real but it is gradual, and it is nothing like the 8x throughput cliff you get from spilling 14 layers to CPU.

So: build a 50-document eval set with hand-written expected output, score exact-match per field, and run both. It is an afternoon and it replaces this entire argument with a number. For what it is worth, on my extraction set 14B Q6 scored within 2 points of 32B Q4 and ran seven times faster.

Contrarian note: try the 32B at IQ3_XXS before you write it off. The IQ quants are much better than the old Q3 was and a 32B at 3 bits sometimes beats a 14B at 6 on tasks that need world knowledge. Extraction from unusual document types is exactly where that can show up. It will fit in 16GB with a short context.

Numbers from a 16GB card so you know what the arithmetic looks like:

• 32B Q4_K_M: 19.9GB of weights. Does not fit, 14 layers to CPU, about 4 tok/s.
• 14B Q8_0: 15.7GB. Also does not fit once you add the cache, about 9 tok/s.
• 14B Q6_K: 12.1GB. Fits with 8k context, 34 tok/s.
• 14B Q4_K_M: 9.0GB. Fits with 32k context, 41 tok/s.

Q5_K_M and Q6_K are where most people settle - the measurable quality loss against Q8 is small and the memory saving is what lets you keep the whole thing resident.

If the schema is stable and you are doing hundreds a night forever, the actual answer in six months is a fine-tuned 7B. A LoRA on 500 hand-corrected examples will beat both of these on your specific documents and run at 90 tok/s. Not a today decision, but stop thinking of the general model as the destination.

Whatever you pick, log the raw output alongside the parsed object. The night you discover a field has been silently null for three weeks, the parsed rows tell you nothing.