How small can an app's AI be on a 4 GB phone, with no network?
We test a small open model, taught only one app's tasks, on an entry-level 4 GB phone. The method, the measurements and their limits are public, so anyone can check them and build on them.
A per-app pipeline, tested on a real phone.
Map the app
Map the actions the app can take and the data behind them.
Build a sandbox
A copy of the app with fake data, so the model can practise every action safely.
Freeze the exam first
An exam from real traffic, frozen before training and kept apart from training data.
Teach a small model
A small open model learns from a larger teacher, with checks.
Add code guards
In our first build, bills, tax and stock are computed by code from the app's own data, not by the model; the AI drafts and the owner approves.
Pass real-device gates
Quantise the model and pass gates on a real low-end device.
The model is a small open model taught only the app's tasks. It starts from Qwen3-0.6B, an open base model under the Apache-2.0 licence, and is retrained only on one app's tasks with examples from a larger 8B teacher, then checked against the frozen exam. It runs with llama.cpp (MIT licence) on the phone's own CPU.
What we have measured, with the layer of each test.
Our first build is the offline edition of the Nesayo pharmacy software. FTL means a real phone in Firebase Test Lab; EVAL means a frozen exam scored on a server. Each row says what it does not show.
Model and speed bench
llama-bench on a Galaxy A05s. The bench did not control the network.
Size only, not quality.
6-bit Q6_K, llama.cpp, 4 CPU cores. No GPU, no NPU.
11.0 tok/s at 768 tokens of context.
Same bench.
Bench conditions: Galaxy A05s SM-A057G, Snapdragon 680 'bengal', 4 GB, Android 15; llama.cpp 620d222 (armv8a); 4 threads; f16 KV, flash-attn 0, repack 0; llama-bench -r 3 (mean of 3); thermal_status 0, battery 31 °C at start.
Whole-app run, a separate run
Galaxy A05s, 4 GB, Snapdragon 680. A test build (31 Oct) of the whole app passed a real-phone run with airplane mode on. No later merged build has passed a whole-app run yet.
The first turn of that run took 13.4 s (right after a print preview); still open.
One run.
The model in memory during the run, not the whole app.
Cold start of one run.
Exam
Scored on a server, not on a phone.
95% CI +4.0 to +24.4. An earlier version of our model (v7), not the current one (v18), scored on a server.
16 of the 40 shops are QA-script traffic, not shop owners.
Full rows, run conditions and sources: benchmarks and evidence.
Open today, and what we plan to publish next.
Plans are not releases. Each one will appear here with its licence when it is out.
Phone benchmark dataset
neykosa/phone-llamacpp-bench on Hugging Face (CC BY 4.0): benchmark rows with their standard deviations, run metadata, and a recipe for measuring a public Qwen3-0.6B file on your own phone. Your numbers describe the public model, not ours: our measured weights are private.
Open on Hugging Face → PublicBenchmark write-up
The full run conditions, including the settings that did not ship.
Read the benchmarks → PublicEvidence page
Every number on our site with its test layer and what it does not show.
Read the evidence →Real-phone test gates
A write-up of what a build must pass on a low-end phone before anyone uses it.
Open eval kit
The exam format and scorer, with synthetic data, so others can freeze an exam for their own app.
Synthetic-data weights
Model weights trained only on synthetic data. Weights trained on real customer chats stay private, because that data belongs to the people who wrote it.
Also: Can a small language model run on a budget Android phone with 4 GB RAM? on dev.to.
What these results do not show.
- Every phone result on this page (FTL) comes from one phone model, the Galaxy A05s. The exam results (EVAL) were scored on a server, not on a phone.
- Energy and battery use have not been measured.
- The offline edition is not yet in users' hands: these are lab results.
- Results describe one app. Another app needs its own exam and its own measurements on its users' devices.
Check it, extend it, or use it.
The dataset card has a recipe for timing a public Qwen3-0.6B file on your own Android phone over adb, and a script that prints your row ready to paste. Send the row, or tell us where the recipe breaks, at hello@neykosa.com. We also welcome reviews of the method from people who research small models or on-device AI.
Building software whose data cannot go to a cloud AI? Our design-partner offer builds the small model for your app's tasks and tests it on your users' phones.