The final payment point is the one to internalise. Once it is paid you are asking for a favour; before it is paid you are negotiating.
Withhold proportionately and say why in writing. Withholding everything for a partial problem is what turns a solvable dispute into a legal one, and it damages your position rather than strengthening it.
Also: pay for what was done properly, promptly. It makes the withheld part obviously about the specific issue rather than about not wanting to pay.