Derin

The costs of local that do not appear in the pitch, and which decide it in practice:

Memory, not compute. A model has to fit in RAM alongside the operating system and everything else. This is what actually limits what you can run on an ordinary laptop, and it is why the demo machine always has a lot of it.

Thermals and battery. Sustained inference makes a laptop hot and drains it fast. Fine for a task, unpleasant as a background service.

Distribution size. Shipping model weights means a large download and an update problem. Users notice.

Support surface. Every machine is different. Cloud has one deployment; on-device has as many as you have customers, and the failure reports are hardware-specific.

That last one is why so many products announce on-device and quietly ship a hybrid.

The honest trade, from shipping both:

Local genuinely wins when:

• The data cannot leave. Regulated material, client confidentiality, recordings of people who did not consent to a third party. This is a real constraint and no amount of contractual assurance satisfies some of it.
• The work is bulk and boring. Transcription, optical character recognition, embedding generation, image resizing. These are well served by small models, and running them locally is free after the download where cloud is per-unit forever.
• Latency matters more than quality. No round trip. For anything interactive at a keystroke, local wins outright.
• Offline is a requirement. Aircraft, sites, poor connectivity.

Cloud genuinely wins when:

• The task is hard. The gap between a small local model and a frontier one is still large for reasoning, long context and code. This is the honest part vendors skip.
• Usage is occasional. Paying per call beats a machine specified for a peak that happens twice a week.
• You need the newest capability, which arrives in the cloud first.