AIOctober 12, 2026· 3 min read
On-device AI is becoming the default for mobile apps
Apple's Foundation Models and Google's Gemini Nano now give every app a private language model that lives on the phone. Here is how we think about on-device versus cloud AI.
By Frogion Team
Two years ago, adding AI to a mobile app meant calling a cloud API and paying for every request. In 2026 both platforms ship a capable language model inside the operating system, and apps can use it with no per-call bill. That changes how a small studio like Frogion designs features.
What Apple offers
The Foundation Models framework is a Swift API to the same on-device model, roughly 3 billion parameters, that powers Apple Intelligence. The @Generable macro returns typed Swift structs straight from a prompt, and the model can call tools your app defines.
At WWDC26 Apple widened the framework considerably for iOS 27:
- Apps can pass images along with text, and the model can call Vision tools such as OCR and barcode reading on-device.
- Dynamic Profiles let an app swap models, tools and instructions in the middle of a session.
- A Language Model protocol lets you plug in cloud models such as Claude or Gemini behind the same API.
- A new Core AI framework lets developers bring their own models and run them on-device or on Private Cloud Compute.
What Google offers
On Android, Gemini Nano runs inside the AICore system service. Apps reach it through the ML Kit GenAI APIs for summarization, proofreading, rewriting, image description, and a general Prompt API. The current model family is Gemma 4 in 2B and 4B effective sizes, and the smaller one runs about three times faster.
Because the weights live in the system, your APK does not grow by hundreds of megabytes. The catch is device coverage: recent Pixels, Galaxy S24 and S25, and the Z Fold and Flip 6 are supported, mid-range phones get partial support, and older devices get none.
On-device, cloud or hybrid?
| Tier | Typical latency | Cost | Good for |
|---|---|---|---|
| On-device model | under 100 ms, works offline | none per request | autocomplete, quick summaries, accessibility descriptions |
| Private Cloud Compute (Apple) | roughly 200 to 800 ms | none per request | personalization, mid-complexity tasks |
| Cloud frontier model | 1 to 5 seconds | billed per token | deep reasoning, fresh knowledge |
Most teams end up hybrid: route to the local model first, fall back to the cloud on unsupported devices or hard tasks, and track cost and latency per tier.
The trade-offs
- Privacy. Nothing leaves the phone, which also helps with the EU AI Act's stance on sensitive data categories.
- Battery. Sustained inference triggers thermal throttling. Apple's Neural Engine is rated at 35 to 38 TOPS on A17 Pro through A19 chips, but you still need to profile across your device range.
- Coverage. On-device features reach flagship users first, so every feature needs a graceful fallback.
What this means for Frogion
Our apps are small, focused utilities. For anything that touches personal data, on-device AI is the natural fit: smart summaries, rewriting and quick suggestions can ship without a server bill and without sending users' data anywhere. We reserve cloud models for the few tasks that genuinely need them.
Sources
- Practical Logix: Mobile App Development in the On-Device AI Era
- Apple Developer: What's new in iOS 27
- Android Engineers: Android + AI in 2026
Cover photo by @steve_j on Unsplash.
