Why small can be the right size
A frontier model is designed to handle a broad range of difficult requests. Many product features do not need that breadth. Summarization, classification, extraction, rewriting and simple tool selection can run on a compact model that responds quickly and does not send every input to a server.
The economics change as well. Once the device and model are available, an additional request has no API token bill. That enables features that operate continuously or offline. The constraints are memory, energy and a smaller margin for ambiguous reasoning.
The platform shift
Apple's Foundation Models framework gives developers access to the on-device model behind Apple Intelligence for language understanding, structured output and tool calling. Microsoft exposes on-device AI capabilities through Windows APIs, and open model families can be optimized for laptops, phones and edge computers.
Platform APIs matter because they manage hardware differences and model delivery. Developers should still record which model version handled a request and test across supported devices. A feature that is fast on a new laptop may be unavailable or battery-intensive elsewhere.
Design a hybrid product
Start with the smallest model that meets a measured quality bar. Route sensitive, offline and routine work locally; send complex or high-value requests to a larger service with user awareness. If the network fails, the interface should explain which capability remains available.
Small models will not end the cloud model race. They will distribute AI across more moments of software. The durable advantage will come from choosing the right location and capability for each task, then making that routing predictable to the user.
Sources & further reading
Social-media activity is treated as a signal of attention, not proof. Product claims are attributed to the linked publisher or announcement.