On-device generative AI can improve privacy, offline reliability, latency, and inference cost. It also introduces constraints such as hardware compatibility, model availability, thermal limits, input limits, and changing APIs.
The safest architecture starts with a product capability, not a model name. “Summarize this note privately” is a capability. A specific AI SDK is only one implementation.
Choose on-device work deliberately
On-device inference is useful when data should remain on the device, the feature must work offline, and the task fits the supported model and device limits.
Cloud inference can remain useful for larger models, broader device support, or tasks that require more context. A hybrid policy should clearly explain when data leaves the device.
Put model APIs behind a boundary
interface TextAssistant {
suspend fun summarize(
text: String,
options: SummaryOptions,
): SummaryResult
}
sealed interface SummaryResult {
data class Success(val text: String) : SummaryResult
data object Unsupported : SummaryResult
data object InputTooLarge : SummaryResult
data class Failed(val cause: Throwable) : SummaryResult
}This keeps the ViewModel independent from a specific model SDK and translates technical failures into states the UI can explain.
Design for uncertainty
AI output is not verified business data. Validate format and length, let users edit or reject results, and test unsupported hardware, cancellation, large input, backgrounding, and malformed output.
Sources: Android AI solutions, Gemini Nano
