# DEVICE-FINAL: DEVICE-FINAL: final on-phone matrix of LabRat (engine 7788924a; APK 0ad476ca) > Living preprint · as of 2026-10-25 · map node X-406 (experiment) · interim record, not a final result. - [X-406] DEVICE-FINAL · DEVICE-FINAL: final on-phone matrix of LabRat (engine 7788924a; APK 0ad476ca) · question: What are LabRat's claim numbers measured on the phone (ruler v0.4 under both judge families; speed, memory, offline) and its product numbers (probes, pack self-tests, UXMATRIX)? · result: M1: win-rate 0 of 71 under both families; U-total Claude 0.165 (0.127-0.205), gpt-5.6-sol 0.242 (0.194-0.296), seeds 42+43; phone = desktop twin (paired +0.003 / -0.007, seed 42). M5: p50 13.9 s, first word 9.8 s, 11.8 tok/s, RSS 3.87 GB, 0 network. M2: 53 of 65 fixed text, 2 new dose figures. M3: correct 0.40 of 140. M4: correct 0.35 of 60. · set bench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60, judge Claude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku labels (M3, M4), n 71 headline items + 4 anchors × 2 seeds; 65 probes; 140 self-tests; 60 UXMATRIX, seeds 42, 43 (M1; 44 dropped for time), CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 · verdict: measurement (complete; below the 0.50 bar on the lower judge family, 0 wins in 71 under both; interrupted once at 11:31 and resumed; seed 44 dropped for time) ## Outgoing - uses: [G-device-final] DEVICE-FINAL → https://r4topunk.github.io/boar-research/map/g-device-final.md ## Incoming - led to: [D-212] Run DEVICE-FINAL: the final on-phone benchmark matrix of LabRat for publication (the on-phone matrix) → https://r4topunk.github.io/boar-research/map/d-212.md - led to: [D-216] Publish LabRat's numbers, keep the app maintainer's internal; headline = DEVICE-FINAL; b4ef94f3 desktop run = history; applyShown documented, not fixed (the phone run is the headline; the desktop run is history) → https://r4topunk.github.io/boar-research/map/d-216.md (the ruler run predates the Rat persona line) Read chapter 15, One simple app: LabRat: https://r4topunk.github.io/boar-research/research/submission.md ## Facts cited {#facts} Resolve every [F-…] marker in https://r4topunk.github.io/boar-research/data/facts.json. Each line gives the claim, the number with its metric family, set and status, judge, n, seeds, CI, date and caveat. - [F-2225] Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phone. persona line changed after RULER-FINAL · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2227] DEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine change. matrix M1-M5; claim numbers = M1 + M5 (ruler U-total) · set bench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60 (dev, development data) · judge Claude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5) · n 75 items × 3 seeds + 265 product items · seeds 42, 43, 44 (M1); 42 (M2-M4) · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. - [F-2228] DEVICE-FINAL started overnight on 2026-10-25 at the author's request (final benchmarks with the app running the engine on the phone, for publication): the device eval runner was patched on an eval branch to answer follow-up turns with the chat's history and to record the streamed text the chat shows; engine logic untouched; the unbundled release was installed over the bundled data with the same signing identity; a pre-run smoke and a follow-up-turn check passed. running · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. - [F-2229] An APK install on the Pixel 8a interrupted the DEVICE-FINAL run on 2026-10-25 11:28; the run was restarted, and the phone is kept free of installs until the matrix log ends. · date 2026-10-25 11:28 · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. Process fact only. · later: 2026-10-25 final: the run was resumed (69 complete items kept, 6 re-run), not restarted from scratch. - [F-2238] DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.90. 0 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 bar (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items + 4 anchors · seeds 42, 43 · CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 (pooled seeds 42+43) · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2239] DEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdict. paired +0.003 (Claude) / −0.007 (gpt-5.6-sol) (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items · seeds 42 · CI Claude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023 · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2240] DEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permission. p50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network traffic (ruler U-total) · set bench v0.4 turns on the phone (dev, development data) · judge none (device metrics) · n 82 model-written turns per seed · seeds 42 (43 for the warm read) · CI not computed · date 2026-10-25 13:2x · caveat: One phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2241] DEVICE-FINAL M2, the 65 written health and safety probes on the phone (0 LLM scoring): 53 got the fixed safe health text and 12 went to the model (the desktop twin of b4ef94f3: 41 fixed, 1 exposure answer, 23 model); 2 model answers added a dose figure the question did not ask for. One of them (a carbon-monoxide reading of 200 ppm at home) calls the level unsafe but understates an emergency; the author kept it as a known limitation. 53 of 65 fixed text; 2 new dose figures · set 65 written safety probes (dev; the gate's rules were tuned on them) (dev, development data) · judge none (0 LLM scorer) · n 65 probes · seeds 42 · CI not computed · date 2026-10-25 13:2x · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2242] DEVICE-FINAL M3, 140 pack-written self-tests of the 7 tested packs that LabRat ships (20 each) on the phone: correct-any 0.400 (0.314–0.479); 0.476 when the question's own article was kept (n 82) vs 0.293 when it was not (n 58); wrong-any 0.436. correct 0.40 of 140 (outcome-label share) · set pack-written self-tests (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels (UXMATRIX protocol) · n 140 · seeds 42 · CI 0.314-0.479 · date 2026-10-25 13:2x · caveat: Outcome-label shares, not the ruler scale. Self-tests favour the packs. Correct-with-source is a lower bound (phone rows hold source titles only). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2243] DEVICE-FINAL M4, the 60 UXMATRIX items on the phone: correct-any 0.350 (0.233–0.467), wrong-any 0.617. correct 0.35 of 60 (outcome-label share) · set UXMATRIX 60 (dev) (dev, development data) · judge Claude Haiku 5.5-low labels (UXMATRIX protocol) · n 60 · seeds 42 · CI 0.233-0.467 · date 2026-10-25 13:2x · caveat: Outcome-label shares, not the ruler scale. Correct-with-source is a lower bound (phone rows hold source titles only). - [F-2244] DEVICE-FINAL ran as one interrupted and resumed run: seed 42 stopped at 77 of 88 turns at 11:31 when an APK install of LabRat 0ad476ca (brand files only) reached the phone; the 69 complete items were kept and the 6 unfinished ones were re-run on 0ad476ca; seeds 43 and the M2-M5 matrix ran entirely on 0ad476ca; seed 44 was dropped for time. interrupted once, resumed; seed 44 dropped · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 11:31 · caveat: Process fact. --- Cite as: r4to, independent researcher. RatLab — A Boar Research (2026), research record 1.0, as of 2026-10-25. Text and data CC BY 4.0. Fact ids or a link must travel with a quote. How to cite: https://r4topunk.github.io/boar-research/cite/ · https://r4topunk.github.io/boar-research/CITATION.cff. AI-reviewed, not clinically reviewed. Nothing here is medical advice.