On-device processing is a selling point again — when does running it locally actually matter?

The honest trade, from shipping both:

Local genuinely wins when:

• The data cannot leave. Regulated material, client confidentiality, recordings of people who did not consent to a third party. This is a real constraint and no amount of contractual assurance satisfies some of it.
• The work is bulk and boring. Transcription, optical character recognition, embedding generation, image resizing. These are well served by small models, and running them locally is free after the download where cloud is per-unit forever.
• Latency matters more than quality. No round trip. For anything interactive at a keystroke, local wins outright.
• Offline is a requirement. Aircraft, sites, poor connectivity.

Cloud genuinely wins when:

• The task is hard. The gap between a small local model and a frontier one is still large for reasoning, long context and code. This is the honest part vendors skip.
• Usage is occasional. Paying per call beats a machine specified for a peak that happens twice a week.
• You need the newest capability, which arrives in the cloud first.

Worth adding the argument that has nothing to do with privacy and often decides it: local means no per-use cost and no dependency on somebody else's business model.

A tool that runs locally cannot raise its prices per call, deprecate the model you depend on, or go out of business and take the capability with it. For anything embedded in a long-lived workflow that is a real form of durability, and it is why some teams choose local for a capability that cloud does better.

One thing worth checking rather than assuming when a product claims on-device: whether it means all of it.

Common and legitimate patterns that are all described as "runs on your device":

• Everything local, no network. The strong claim.
• Local for the routine path, cloud for the hard one, with a toggle.
• Local for the media, cloud for the reasoning about it — very common in editing tools.
• Local processing, cloud sync of the results.

These have completely different privacy properties and all four get the same sticker. The test is the boring one: run it with the network off and see what still works, then watch the connections when the network is on.

If privacy is the actual reason you are choosing it, that ten minute check is the whole due diligence.

The costs of local that do not appear in the pitch, and which decide it in practice:

Memory, not compute. A model has to fit in RAM alongside the operating system and everything else. This is what actually limits what you can run on an ordinary laptop, and it is why the demo machine always has a lot of it.

Thermals and battery. Sustained inference makes a laptop hot and drains it fast. Fine for a task, unpleasant as a background service.

Distribution size. Shipping model weights means a large download and an update problem. Users notice.

Support surface. Every machine is different. Cloud has one deployment; on-device has as many as you have customers, and the failure reports are hardware-specific.

That last one is why so many products announce on-device and quietly ship a hybrid.