RatLabA Boar Research
Closed recordas of what’s still open →

The research · Act II, Pushing the engine · Chapter 07

A phone on a desk, and in a hand

The desktop stand-in was trustworthy for answer quality, and the frozen build matched it on real phones, but the phone exposed bugs and memory limits the desktop hid. The deep tier never ran on a phone (the author dropped it from delivery on 2026-10-21); the 8 GB phone runs of 10-20/21 changed the main engine (chapter 11).

2026-10-02 → 10-22← In the article: §7 The app: LabRat

Testing on a desk

Running a test on a phone takes hours. Running the exact same code on a desktop takes minutes. We did most of the work on the desktop, and then checked it was telling the truth.

The phone fights back

A phone gets hot, runs short of memory and hides different bugs. On one 8 GB phone the model was killed while loading because a speed trick quietly made a second copy of its weights. On another, every question re-read the whole prompt because of a bug in our own code that the desktop could not show.

Same answers, slower

On a real phone, the frozen build on a 12 GB-class phone in airplane mode answered all 72 test questions and scored exactly what the desktop scored: 0.529 and 0.529. A typical answer took about 31 seconds. On a real 12 GB-class phone, in airplane mode, the frozen build gives the same judged quality as on the desktop.phone raw 0.529 (0.481–0.575) vs desktop 0.529; floor-adjusted 0.403 vs 0.402; paired −0.000 (−0.026…+0.028); 72/72 answered; fast-tier p50 30.7 s; app memory (PSS) peak 5.88 GB during model load, then ≤ 4.23 GBSetv0.1Statuspublic devJudgeClaude Haiku 5.5-low (L1 triage judge, N = 4, both orders)n57Seeds1 runDate2026-10-14CaveatHaiku-judged level; not comparable with Opus-judged ratios. The device has 10.7 GiB usable RAM.Sourceresearch record arms/device-x6/DEVICE-X6.md:15-18research record arms/device-x6/DEVICE-X6.md:27research record arms/device-x6/DEVICE-X6.md:27-30research record arms/device-x6/DEVICE-X6.md:27-28research record arms/device-x6/DEVICE-X6.md:13Open the data file

The big model has not been on a phone

The 35-billion-parameter model never ran on a phone. A technique that streams the model's experts from storage kept its working memory under 3 GB on a desktop, and was even built for Android, but nobody measured it on a phone before the big model was dropped from the plan.

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

Phones: parity, latency, memory

Same quality as the desktop, 3.4× slower, 4.2 GB after load

We trusted the desktop only after phones agreed with it. Cut 7 on a 12 GB-class phone (MediaTek MT6897, 10.7 GiB usable, Android 16, airplane mode, no INTERNET permission) answered 72/72 with 0 timeouts. Tier routing matched 60/60, instant answers were byte-identical 16/16, kept source titles matched 59/60, and model text was identical on 3/44 (sampler differences). Quality on the 57 public v0.1 items under the Haiku L1 judge: raw 0.529 (0.481–0.575) on the phone vs 0.529 on desktop; paired −0.000 (−0.026…+0.028). Fast tier: p50 30.7 s, p90 48.4 s, TTFT p50 18.5 s, decode 5.5 tok/s, output p50 64 tokens. Memory: PSS 5.88 GB peak during model load, then p50 3.64 GB, max 4.23 GB. Thermal status stayed 0.

Gotchas

  • ARM weight repack. llama.cpp's repack copied ≈ 2.3 GiB of weights into anonymous memory and the OS killed the 4B on an 8 GB phone while loading; disabling it dropped desktop max RSS 5.1–5.4 → 3.41 GB. The fix (repack-aware fit, Android only) is not in the frozen build The 4B weight repack on ARM duplicates ~2.3 GiB in anonymous memory.Probe A: anon 3.25 → 0.88–0.91 GB, max RSS 5.1–5.4 → 3.41 GB with no-repackDate2026-10-02Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatmacOS desktop probeSourcenot in the public repositories yetOpen the data file.
  • The desktop shim hid our own bug. A KV-cache session save path was a no-op on desktop, so every phone question re-prefilled ≈ 430 + 300 tokens (35–41 s + 20–28 s) KV-cache save bug cost on the A15.gate prefix 35–41 s + answer prefix 20–28 s re-prefilled per question at ~12 tok/s; fix est. −55 to −65 sDate2026-10-05Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatour own regression, not upstream (QUEUE:649)Sourcenot in the public repositories yetOpen the data file.
  • q8_0 KV cache is non-inferior (Δ −0.0077, −0.0232…+0.0062) and halves reader KV 576 → 306 MiB q8_0 KV cache is non-inferior on quality.Δ −0.0077 (−0.0232…+0.0062); reader KV 576 → 306 MiBSetv0.1 devStatusdevJudgeOpusn60Seeds2Date2026-10-05Caveatdesktop slower; device-day candidate onlydevice candidateSourcenot in the public repositories yetOpen the data file.
  • Pure-JS zstd was ≈ 76 % of retrieval time on the phone; native zstd ≈ 80× faster Retrieval JS on the phone: pure-JS zstd decompression ~76 % of retrieval time; native zstd ~80× faster499 ms of 677 msJudgen/aDate2026-10-02Kinda record, not a measurement (set, status, n, seeds: not recorded)Caveatled to native zstd arm (device-only gains)Sourcenot in the public repositories yetOpen the data file.

The deep tier and phones: what we know, and what was never measured

Never run on a phone: two engines, one projection, an Android build that was not measured

The deep tier only ever ran on desktops. Two on-device paths were tested there. bmoe, a streamed-MoE engine, is not token-identical to llama-server (greedy 0/10) but judged-equivalent on identical contexts (0.623 vs 0.634, Δ −0.012, CI −0.045…+0.020; v0.1 dev, n 57, 1 seed, Opus); its macOS session peaked at 11.7 GiB. An expert-streaming fork (LES) runs the same 35B at 14–17 tok/s with 2.75–2.81 GiB peak RSS on an M4 desktop CPU with experts read from SSD and page cache, and is judged-equivalent too (0.625 vs 0.634, Δ −0.010, CI −0.042…+0.018). On a phone it would be flash-bound: about 320 expert reads and 314 MiB per token, so roughly 4 tok/s on UFS 3.1 and 6 tok/s on UFS 4.0 including compute (I/O-only upper bounds 4.8 / 8.0). Using third-party public phone rates, a deep + 512-thinking answer would take about 1.9–2.8 minutes on a Pixel 8 Pro CPU. None of this was measured on a phone. The expert-streaming engine was cross-built for Android arm64 on 2026-10-19 Expert-streaming engine cross-built for Androidbuilds; phone numbers unmeasuredDate2026-10-19Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatno phone run in this sliceSourcenot in the public repositories yetOpen the data file, and then the author's 2026-10-21 decision (no 35B deep mode) dropped both phone fronts (“Deep tier on a 12 GB-class phone”, “Expert-streaming engine on Android” in ). The projection stays a projection.

The lite tier and long sessions

1.7B for small phones: faster, safe-routed, weaker reader (the 10-16 lite-tier view; superseded by T14)

For 4–6 GB phones we tested Qwen3-1.7B (1.1 GB file, RSS ≈ 2.3 GB vs ≈ 4.0 GB for the 4B; desktop prefill 2.5× and decode 2.2× faster) behind the same safety engine; health routing matched the 4B 124/124. With X2b it matches the plain 4B only on pack-matched dev items (−0.007, −0.031…+0.018, Haiku); on the claim kit it is 0.111 below the 4B and X2b is inert there; at identical contexts it reads worse (−0.070, −0.094…−0.044, n 97). Phone speed for the lite tier was unmeasured as of 2026-10-17; on 2026-10-20 the 1.7B became the main 8 GB engine on measured phone speed (T14). Separately, long sessions do not degrade the 4B: turns 6–15 score 1.136 (0.993–1.320) of BOAR's own standalone answers (Haiku L1, 80 agent-written questions), but latency grows to about 22 s at turns 14–15 vs 8 s standalone on desktop.

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

§6.6 Device parity and speed

The frozen build (cut 7) on a 12 GB-class phone (MediaTek MT6897, 10.7 GiB usable, Android 16, airplane mode, no INTERNET permission) answered 72/72 items; on the 57 judged public v0.1 items, judged by Haiku L1, phone raw 0.529 (0.481–0.575) equals desktop 0.529, floor-adjusted 0.403 vs 0.402, paired −0.000 (−0.026…+0.028). Routing tier matched 60/60 and instant answers were byte-identical 16/16; model text was identical on 3/44 items (sampler differences). Earlier, cut 6 on an 8 GB phone scored 0.561 (0.507–0.617) vs desktop 0.558/0.560 under Opus judge.v1-med; the two phone studies are not comparable in level. Phone fast-tier p50 is 30.7 s (desktop 7.5 s on an M1 Pro; ×3.4); PSS peaks at 5.88 GB during load and stays ≤ 4.23 GB after. The deep tier never ran on a phone and was not delivered (author, 2026-10-21); two on-device engines are judged-equivalent to the desktop server on identical contexts (bmoe −0.012, −0.045…+0.020; expert streaming −0.010, −0.042…+0.018; v0.1, 1 seed, Opus), and a deep + 512 answer is projected at ≈ 1.9–2.8 min on a Pixel 8 Pro CPU from third-party rates.

§6.7 Multi-turn and lite tier

Over 10 five-turn sessions and a 15-turn extension (80 agent-written questions, Haiku L1, reference = the system's own standalone answer, not the frontier), in-session answers score 1.136 (0.993–1.320) of standalone on turns 6–15; latency rises to ≈ 22 s at turns 14–15 vs ≈ 8 s. A 1.7B lite reader with X2b equals the plain 4B only on pack-matched items (−0.007, −0.031…+0.018); on the claim kit 1.7B − 4B = −0.111, and at identical contexts the 1.7B reads worse (−0.070, −0.094…−0.044, n = 97). An earlier internal summary ("the lite gap is reach, not reading") was retracted on this evidence. §6.8 reports the 1.7B as main engine from 2026-10-20.

Badge rows the 7 public fact rows this depth cites; a number here without a chip comes from a row that is not public yet (see /data/)
Claim and valueSetStatusJudgenSeeds95 % CIDate
The on-device MoE engine (bmoe) is not token-identical to the desktop server but is judged equivalent on identical contexts.0.623 vs 0.634, Δ −0.012 (−0.045…+0.020); greedy token-identical 0/10v0.1public devClaude Opus (judge.v1, both orders)5712026-10-07
Cut 7 on the POCO X6 Pro (MT6897, 10.7 GiB, Android 16, airplane, no INTERNET permission).72/72 success, 0 timeouts, logcat cleanv0.1 public + safetypublic devnot recorded72not recorded2026-10-14
Multi-turn sessions do not degrade answers over 5 turnsturns 2–5 ratio in-session/standalone 1.397 (1.187–1.673); drift +0.537 (+0.208…+1.155)agent-written devdevHaiku L1not recorded12026-10-14
Long conversations do not degrade answers up to turn 15.turns 6–15, in-session vs standalone: 1.136 (0.993–1.320); self-contained items 1.017 (0.950–1.095); latency ≈ 22 s vs 8 s at turns 14–15LONGSESSION dev sessionsdev (agent-written)Claude Haiku 5.5-low (L1 triage judge, N = 4, both orders)80 agent-written questions12026-10-14
1.7B + X2b vs plain 4B on pack-matched items−0.007 (−0.031…+0.018)180 self-tests devdevHaiku180not recorded2026-10-16
Lite gap persists with identical contexts1.7B X2b − 4B X2b −0.070 (−0.094…−0.044)180 self-tests devdevHaiku97not recorded2026-10-16
Retracted summary: "the small-model gap is reach, not reading"at identical contexts 1.7B − 4B = −0.070 (−0.094…−0.044)180 self-tests (dev)devHaiku97,12026-10-16

In one sentence

The phone matches the desktop on quality; it is just slower, and the big model never left the desktop.

Turning point

2026-10-14 19:26, DEVICE-X6: the frozen build on a 12 GB-class phone gives the same judged quality as the desktop.

On the calendar

32 dated events · 2 Oct → 24 Oct

  1. 2 Oct · 06:10Strategy memo (high-tier model, one call): Android-only, three fronts; stop iPhone/1.5B and prompt work
  2. 5 OctFirst real-phone run of a frozen build on a volunteer tester's 8 GB phone (smoke, 9 items)
  3. 5 OctFirst real-phone run (cut 3 smoke, Galaxy A15 8 GB): 9/9, tree hash = desktop, latency 52–147 s
  4. 5 OctA15 diagnosis: KV-cache save path bug (our own regression) re-prefills ~430+~300 tokens per question
  5. 6 OctPhone (8 GB) full run, cut 6: 73/73, phone 0.561 = desktop 0.558/0.560
All 32 events
  1. 2026-10-02iPhone 13 run matches the desktop proxy (0.394 vs 0.395)
  2. 2026-10-0235B memory check in a 6,750 MiB (≈ 6.6 GiB) Linux container: 10/10, 0 OOM
  3. 2026-10-02Bench v0.2 started; iPhone device run
  4. 2026-10-02Root cause of a 4B kill on an 8 GB moto phone: llama.cpp ARM weight repack copies ~2.3 GiB into anonymous memory
  5. 2026-10-02Probe A: no-repack drops 4B max RSS 5.1–5.4 → 3.41 GB on desktop
  6. 2026-10-02iPhone 13 repack probe: llama.rn does not repack on iOS; fit-repack stays Android-only
  7. 2026-10-02 06:10Strategy memo (high-tier model, one call): Android-only, three fronts; stop iPhone/1.5B and prompt work
  8. 2026-10-02 18:30Own 8 GB Android: 4B killed by the OS while loading; root cause = ARM weight repack
  9. 2026-10-03Deep tier D1 (on-device MoE engine build) done
  10. 2026-10-03Deep tier D1: pinned build of the bmoe streamed-MoE engine (Android payload 12.4 MB); deep work paused for token budget
  11. 2026-10-03Ship cut 1 / 1b built (health cards, gate cold-load, travel calc, CO hazard, device detector)
  12. 2026-10-05First real-phone run of a frozen build on a volunteer tester's 8 GB phone (smoke, 9 items)
  13. 2026-10-05Cut 4 = cut 3 + KVFIX + INSTANT-FRAG (+ INSTANT-SUBJ)
  14. 2026-10-05No-network measurement APK (INTERNET permission removed by construction)
  15. 2026-10-05Cut 4 (KV-cache fix + instant fragment) built; cut 5 (lite gate skip) adopted as device candidate
  16. 2026-10-05First real-phone run (cut 3 smoke, Galaxy A15 8 GB): 9/9, tree hash = desktop, latency 52–147 s
  17. 2026-10-05A15 diagnosis: KV-cache save path bug (our own regression) re-prefills ~430+~300 tokens per question
  18. 2026-10-06Phone (8 GB) full run, cut 6: 73/73, phone 0.561 = desktop 0.558/0.560
  19. 2026-10-06A15 latency budget: warm p50 68.7 s model-path answer predicted for cut 4
  20. 2026-10-07VENUEGPS-2: venue items 0/7 → 7/7 with device location + area tile
  21. 2026-10-07DEEPPAR-L1: on-device MoE engine (bmoe) judged-equivalent to llama-server (Δ −0.012) → TRANSFER
  22. 2026-10-07VENUEGPS-2: venue items 0/7 → 7/7 with device location + area tile (plumbing proof)
  23. 2026-10-10Deep tier handoff note: config + engine changes needed; no 12 GB device
  24. 2026-10-10 21:05Decision note with defaults sent to the author (claim vehicle, bar scale "raw formal + floor-adj + win-rate", no v0.3, no device relay)
  25. 2026-10-10 21:27Cut 7 no-network measurement APK built and audited
  26. 2026-10-10 21:27Measurement-app ("BOAR Bench") cut 7 no-network APK built and audited: no INTERNET permission
  27. 2026-10-10 21:40Android chat wired to the measured engine on a branch; identity check 20/20
  28. 2026-10-13 21:14Phone tester package for a 12 GB-class phone built and shared via a link folder
  29. 2026-10-14Cut 7 on a 12 GB-class phone (POCO X6 Pro): device = desktop
  30. 2026-10-16 18:13LESPORT: expert-streaming fork runs the 35B in 2.8 GiB RSS on desktop CPU, judged parity TRANSFER
  31. 2026-10-24 18:45BoarLite on a Pixel 8a in airplane mode: 3 GB unpacked in 14 s, 0 bytes network, PSS 3.09 GB
  32. 2026-10-24 19:12Candidate APK on the Pixel 8a: model ready at 18 s, first answer 11.9 s at 14.9 tok/s

Related nodes on the map

This chapter on the map →

Concepts

Experiments

Decisions

.md