A stock answer on a 4 GB phone, offline: 4.25 s.
The offline edition of our first build answered a stock question in 4.25 s on a Galaxy A05s in airplane mode. The first turn of that run took 13.4 s (right after a print preview); still open. FTL
This page is a selection of our measured results: each with its layer, its conditions and its limits. FTL: lab-tested on a real phone in Firebase Test Lab. EVAL: scored on a frozen test on a server. SIMULATED: synthetic shops, offline build on the VM; not device proof. Published 4 Nov 2026.
Our full result files are not public; file names are given so a row can be traced when you ask us. A filtered copy of the speed-bench results is public: neykosa/phone-llamacpp-bench on Hugging Face.
First build: Nesayo (pharmacy software, nesayo.com). The live Nesayo product uses cloud AI; the offline edition runs our model inside the phone, is tested on real phones in a device lab (Firebase Test Lab), and a shop pilot is next.
How did the whole app run on the test phone?
One run. Galaxy A05s, 4 GB, Snapdragon 680. Android 15, airplane mode, run PASSED.
- Stock answer, offline FTL
- 4.25 s. The first turn of that run took 13.4 s (right after a print preview); still open
- Bill from a sentence FTL
- 5.76 s
- Model memory FTL
- 673 MB
- First launch FTL
- 18.8 s. cold_start_ms 18786
Result file: ftl_final_whole_app_a05s_26efc64a1_20261031.json.
How fast is the model on its own?
A separate llama-bench run, not the whole-app run above. Network not controlled (not airplane mode).
- Writing speed FTL
- 14.8 tok/s. f16 KV, flash-attn 0, depth 0, tg64. This is the setting that shipped. At 768 tokens of context: 11.0 tok/s
- Reading speed FTL
- 25 tok/s. pp128, f16 KV, no flash-attn: 25.45 ± 1.35 (the shipped setting)
- Model size FTL
- 489 MB. qwen3-0.6b-v18-Q6_K.gguf
- Parameters FTL
- 0.6B
- Precision / runtime FTL
- 6-bit Q6_K, llama.cpp, 4 CPU cores. llama.cpp 620d222
Bench run conditions: Galaxy A05s SM-A057G, Snapdragon 680 'bengal', 4 GB, Android 15; llama.cpp 620d222 (armv8a); qwen3-0.6b-v18-Q6_K.gguf 489 MB; 4 threads; f16 KV, flash-attn 0, repack 0; llama-bench -r 3 (mean of 3); thermal_status 0, battery 31 °C at start.
Result file: ftl_raw_a05s_v18q6_fa_kv_20261026.json. The shipped-setting rows with their standard deviations are on our benchmarks page and in that public dataset.
What have test builds shown, not yet in the shipped app?
Two results from test builds. Neither describes the shipped app.
- Wake-up on reopen FTL
- 11.8 s → 0.51 s (23x). warmed KV state saved on the phone, sha256-checked; replies byte-identical; test build (kvwarm APK), not yet the shipped APK
- Own Arm kernel patch FTL
- +34 % prompt reading speed. 36.71 → 49.14 tok/s, Q8_0 model, armv8.0 NEON repack, no mmap; +158 MB memory; test build
Result files: warmup_kv_restore_20261101.json and ftl_raw_a05s_q8_neon_nommap_20261101.json.
How did the small model score against the model that taught it?
On a frozen test scored on a server. This compares v7, not the shipped v18.
- 0.6B vs its 8B teacher EVAL
- +13 pts (95% CI +4.0 to +24.4). v7, not the shipped v18; frozen real test, 40 shops, shop-clustered, scored on a server
- Exam size EVAL
- 322 questions, 40 shops. frozen real-traffic test; the +13 pts compares v7 (0.6B) with the 8B teacher. 16 of the 40 exam shops are QA-script traffic
- Exam kept out of training rule
- The real-message TEST split is frozen and never used for training. Policy, not a separate audit
Result file: r1_clustered.json.
Which features work offline, and where does the build fall short?
Synthetic shops and an offline build on the VM. These rows are not device proof.
- Features working offline SIMULATED
- 95 of 106. 95 PASS, 4 DEGRADED, 5 FAIL (photo/PDF stock entry, prescription scan), 2 NEEDS-INTERNET
- Feature checks SIMULATED
- 235 passed, 0 failed. 3 not offered + 4 open out of 242; synthetic shops; offline build on the VM
Result files: parity_walk_346c4eadb.json and feature_correctness_26efc64a1-trackF.json.
What do these results not establish?
- The accuracy of the shipped model: the +13 pts result compares v7, not the shipped v18.
- The speed-bench figures as an offline result: that run had its network not controlled (not airplane mode).
- The wake-up and kernel-patch gains in the shipped app: those two rows come from test builds.
- Feature behaviour on a phone: the feature rows are SIMULATED.
- Energy use, or a cloud latency or cost comparison: neither is measured.
- Your app's accuracy or timings: these rows describe the Nesayo offline edition, and your app would need its own measurements.
No paying customer, and no pilot running. A shop pilot is next.
Want these rows measured for your app?
Our design-partner offer is for software companies whose app handles data it cannot send to a cloud AI. We build the small model for your app's tasks and test it on your users' real phones.
Start with what your software does, one task you want the model to handle and your target devices.