Litert
LiteRT Android Deployment Guide: Ship Local Models Without Play Store Delays
A practical guide to deploying LiteRT models on Android. Covers packaging, background downloads, device-tier routing, rollback, NNAPI variability, and when LiteRT is the right runtime.
LiteRT Android Deployment Guide: Ship Local Models Without Play Store Delays
LiteRT is the default Android runtime for a lot of local AI work because it solves the first problem cleanly: get a model onto the phone and run it with a small, well-supported native stack. The trouble starts after the demo. Production teams do not fail because LiteRT could not execute a tensor. They fail because the model update path is welded to the APK, device behavior varies by SoC, and a rollout plan never existed.
If your Android feature already works with LiteRT, the next question is not whether to keep benchmarking. The next question is how to ship model updates, segment devices, and recover from a bad artifact without waiting for another Play Store release.
This guide covers that layer.
What LiteRT is good at
LiteRT is the Android-native path for small and medium on-device models:
- image classification, detection, and segmentation
- speech and audio models
- embedding and ranking models
- task-shaped LLM deployments that fit mobile memory budgets
It is especially attractive when you need:
- a compact runtime
- APK/AAB-friendly integration
- good ARM CPU performance through XNNPACK
- hardware offload when the device’s acceleration path cooperates
That last qualifier matters. Android acceleration is real. It is also inconsistent enough that you should design for variance instead of assuming every flagship benchmark generalizes to the fleet.
The production gap
The default LiteRT deployment story is simple: put the .tflite model in the app bundle and load it locally. That works for one version. It breaks down when you need to operate the model as part of a product.
Bundled-only deployment creates four recurring problems:
- App-store-gated model updates. Every model fix waits on the next app release.
- No selective rollout. You cannot send a new model to 5% of supported devices first.
- No clean rollback. A bad model often means shipping another app update.
- One artifact for unlike hardware. Mid-range Android phones and top-tier devices do not want the same model profile.
If the model is important enough to update, it is important enough to separate from the binary.
A deployment shape that holds up
For Android, the stable pattern usually looks like this:
App install
↓
Baseline UI + safe fallback path
↓
Model manifest fetch
↓
Background LiteRT model download
↓
Integrity check + device-tier match
↓
Activate model locally
↓
Keep previous version cached for rollback
That buys you smaller installs, staged activation, and a way to keep the feature alive while the model catches up.
Package the model like a release artifact
Do not treat the .tflite file as an anonymous blob. The package should carry enough metadata that your app and rollout system can make policy decisions without guessing.
At minimum:
- model ID and version
- expected RAM and disk footprint
- minimum app version
- supported device tiers
- quantization and input-shape details
- checksum for integrity verification
- rollback target
If you cannot answer which exact LiteRT artifact ran on a user’s device, you do not have a deployment system. You have a file download.
Device tiers matter more on Android than people want them to
Android is not one runtime surface. It is a range:
- 6 GB devices that can support a compact local path
- 8 GB devices that can handle a more useful model and moderate context
- 12 GB+ devices where local features feel materially better
- OEM builds that expose acceleration differently even with similar specs
The practical move is to define cohorts before rollout:
- small tier for conservative artifacts and shorter prompts
- mid tier for the default production model
- high tier for larger artifacts or longer context budgets
- unsupported tier for cloud or server fallback only
If you skip this, the local feature is being optimized for your QA phones, not your user base. Pair this with the mobile LLM memory budget guide if the model is text-generative.
NNAPI, GPU, and CPU fallback
LiteRT’s hardware story is strongest when you plan for a reliable CPU path first and treat acceleration as an upgrade, not an entitlement.
The usual stack:
- XNNPACK CPU as the dependable baseline
- NNAPI / vendor acceleration where the device actually benefits
- fallback logic when the accelerated path regresses latency or stability
The mistake is building the launch promise around the best-case device path. A rollout needs to survive the median device, not just the benchmark box.
Background downloads need a product contract
Android teams routinely get the model onto the phone but never define what the user sees while that is happening.
Decide this before launch:
- Does the feature wait for the download or degrade gracefully?
- Is the feature usable with a smaller bootstrap model?
- What happens on metered networks?
- What happens when storage is low?
- When does the app evict old artifacts?
If those answers are vague, users will discover them for you in Play Store reviews.
Rollouts need model gates, not just app gates
Crash-free sessions are not enough for a LiteRT model rollout. You also need model-specific gates:
- download success rate
- activation success rate
- P95 time to first result
- OOM or memory-pressure rate
- battery/thermal regressions where available
- local-success vs fallback rate
This is the same logic behind the local AI rollout checklist and the broader on-device model rollout playbook: the model has its own release lifecycle even when the app binary does not change.
When LiteRT is the right Android runtime
LiteRT is a strong fit when:
- the team is already Android-first
- the model fits a bounded mobile budget
- you want a compact native integration path
- the feature can be expressed cleanly as local inference on one device
It is a weaker fit when:
- you need broad cross-platform runtime symmetry more than Android-native convenience
- the feature depends on a heavier local model than your Android cohort can hold
- the operational problem is now rollout and routing rather than raw inference
If the model came through MediaPipe tasks, use the MediaPipe deployment guide. If the architecture is still being chosen, start with What Is On-Device AI?.
A sane first rollout
The low-drama path is narrower than most teams want:
- Start with one Android feature, not a whole assistant.
- Define supported RAM tiers up front.
- Ship the model outside the app binary.
- Keep the previous artifact cached.
- Expand traffic only after device-tier metrics pass.
- Route unsupported hardware elsewhere instead of pretending the local path is universal.
That is less exciting than “runs on Android.” It is also how the feature stays live after launch.
Getting started
- Read On-Device Model Rollout Playbook if the local feature already works and now needs a controlled release plan.
- Read MediaPipe + Octomil: On-Device ML Deployment for Android and iOS if your Android stack starts from MediaPipe tasks instead of direct LiteRT integration.
- Read Mobile LLM Memory Budget Guide if your rollout risk is RAM, context growth, or device coverage.
- Download Octomil if you want a clean local workflow now.
FAQ
Is LiteRT the same as TensorFlow Lite?
LiteRT is the current Android-facing branding for the mobile runtime path many teams knew as TensorFlow Lite. The deployment questions are the same: artifact packaging, hardware variability, and how you update models once users already have the app.
Should I bundle the LiteRT model inside the APK?
Only if the model changes rarely and the product can tolerate Play Store-gated updates. For anything you expect to tune, separate the model from the app release so you can stage, pause, and roll back independently.
Is NNAPI enough for production acceleration?
Sometimes, but not uniformly. Plan around a reliable CPU baseline and treat NNAPI or vendor acceleration as a device-tier improvement. Android hardware diversity is the reason rollout policy matters.
How do I roll back a bad LiteRT model?
Keep the previous version cached, mark the new artifact inactive in your control layer, and retarget the affected cohort without waiting for a new app build. If rollback requires a Play Store release, the model is still coupled to the app.
When should I use MediaPipe instead of direct LiteRT integration?
Use MediaPipe when the task abstraction is already a good fit for your feature and you want its packaged pre/post-processing. Use direct LiteRT integration when you need tighter control over the runtime surface or a more custom model path.