Can a phone with no internet answer like the web plus a frontier AI?

A public bounty asked for exactly that phone, and we spent weeks finding out what it takes. The short version: a small model is only as good as the page you hand it. On questions written from its own library, when search put the right article in front of our small model, it was right about two times in three pack-written · 1 seedCorrect-any when own article is kept vs not0.65 (n 98) vs 0.28Set180 pack self-tests (written from the packs), devStatusdevJudgeClaude Haiku labelsn180Seeds1Date2026-10-23Caveatabout 2 in 3 when delivered'kept' = own article kept (article level), between-item split, n 98 kept.Sourcenot in the public repositories yetOpen the data fileClose. So we stopped trying to make the model smarter. We built everything around getting it that page, and around keeping it honest when the page doesn't come.

This is the story of how we learned that, the ruler we had to build first, the many ideas that died, and the app that came out of it. As of 2026-10-25 it is not shown to be half as good as frontier AI with web: on the phone it won 0 of 71 questions against the reference, scoring 0.165 and 0.242 under the two judges where 0.50 is the bar phone · 2 seedsDEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.900 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 barSetbench v0.4 (exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references)n71 headline items + 4 anchorsSeeds42, 43Date2026-10-25 13:2xCaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen. One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose.

#1The question

The bounty grew out of a public post by Vitalik Buterin asking for an offline research tool on a phone public postPublic post (by Vitalik Buterin) asks for an offline phone research tool ">50 % as good as internet + frontier AI"; bounty #31 is later opened on poidh, inspired by itDate2026-09-17Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose. Poidh bounty #31 was opened later, inspired by that post. He did not issue it and is not affiliated with it, though his public confirmation that a build meets the target would win it bounty textBounty re-read: winning is a public confirmation, not a numeric benchmarkDate2026-10-02Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose.

The rules are concrete. An Android phone with at most 12 GB of RAM, no connection and no Google Play Services bounty textThe bounty's hardware envelope.≤ 12 GB RAM, ≤ 50 GB total storage, offline, no Play Services, GrapheneOS-compatibleDate2026-10-02Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatGrapheneOS not yet run on hardwareSourcenot in the public repositories yetOpen the data fileClose. Answers "more than 50 % as good as internet plus a frontier AI". The bounty page gives a public decision date of 2026-10-31 bounty pageDeadlines: the bounty page (swept 2026-10-13) gives a final decision by 2026-10-31, and the issuer has said there is no fixed deadline; 2026-10-20 was the team's own closing-sprint deadline.2026-10-31 (bounty page); 2026-10-20 (closing sprint)Date2026-10-14Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatUNKNOWN whether the bounty itself moved to 10-20; call 10-20 the team's closing-sprint deadline unless the author confirms.Sourcenot in the public repositories yetOpen the data fileClose. We chose to work in English only team scopeSingle goal = win the bounty; English only; no more work inside the maintainer's app; build a dedicated measurement app — why: focus — outcome: PT/ES lines parked; BOAR Bench builtDate2026-10-03Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose.

That target hides a hard question. Half of what? Measured how, by whom, on which questions? If you grade a student's essay against a professor's, "half as good" depends on who grades and what counts. So before improving anything, we had to build a ruler we could trust.

#2The ruler

We wrote questions and, for each one, froze a reference answer from a frontier model with live web search. The references were hashed and never regenerated, so every later run faces the same opponent methodReferences = Claude Opus + web search, frozen once, hashed; never regenerated (why: a fixed denominator makes runs comparable)Date2026-09-30 → 10-02Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatoutcome: same refs used for every v0.1 number; regenerating = not comparableSourcebenchmark README.md:48-50benchmark TRACE-FORMAT.md:79Open the data fileClose.

An AI judge then reads our answer and the reference side by side. Links and citation marks are stripped first, so it cannot simply spot the answer that came from the web methodBlinding strips think blocks, trailing "Sources:" lists, URLs, markdown links, [n] markers and bold before judging.1 functionSetallKinda record, not a measurement (status, judge, n, seeds, date: not recorded)Caveatstripping sources also removes web provenance signals from the referenceSourcebenchmark scorers/vendor/eval-lib/judge-core.mjs:5-20Open the data fileClose. It scores both from 1 to 5 on correctness, completeness and usefulness rubricThe judge scores each answer 1-5 on three axes: correctness (a material wrong claim caps it at 2; an uncorrected false premise at 1), completeness, usefulness.3 axes, 1-5Setall setsJudgejudge.v1Date2026-10-02Kinda record, not a measurement (status, n, seeds: not recorded)Sourcebenchmark scorers/vendor/eval-lib/judge-core.mjs:40-44Open the data fileClose. Then we swap the order and ask again. If its pick flips, the question counts as a tie ruleCombining the two orders: same winner keeps it, disagreement becomes a tie; rubric scores are averaged.ruleSetallKinda record, not a measurement (status, judge, n, seeds, date: not recorded)Sourcebenchmark scorers/vendor/eval-lib/judge-core.mjs:93-98Open the data fileClose.

Two choices matter most.

Two judges from different families. Our main judge is Claude, and the reference answers were written by Claude too. A judge might favour writing from its own family, and a second Claude judge cannot catch that method checkChanging the judge to another Claude model: Fable second judgemoves the ratio < 0.01Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatcitation fix (l.27 is a table header); why: does not address self-preferenceSourcenot in the public repositories yetOpen the data fileClose. So we added a judge from OpenAI, gpt-5.6-sol through the Codex CLI. Run after the fact, it scored both systems it re-judged 0.08 to 0.14 higher, but kept the same ranking and almost no wins dev-style · 2 seeds · run after the factA judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate.4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01SetV-paraStatusdev-styleJudgecross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered)n51 (408 judge calls)Seeds2Date2026-10-16CaveatPost-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline.Sourcenot in the public repositories yetOpen the data fileClose. A rule fixed in advance says the stricter judge sets the headline ruleClaude judge stays the headline (lower of two judges); judge family named next to every ratio; win/tie/loss leads — why: analysis-plan rule: when judges differ by > 0.05, the lower is the headline — outcome: claim v4.2.xDate2026-10-16/17Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose.

Rules before results. Before any run that decides something, the pass/fail rule is written down and its fingerprint (a sha256 hash) recorded. An audit matched 65 of 65; an outside critic later found 16 of 17 matching auditCriterion hashes verified: 65/65 on 2026-10-10; an outside critic found 16 of 17 matching on 2026-10-16.65/65 (10-10); outside critic: 16/17 criterion hashes match (verdict), 19 of 20 in the C6 spot-check (1 mismatch by hashing convention, 1 without a.sha256 sidecar)Date2026-10-10, 2026-10-16Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatsome early freezes attested by mtime onlySourcenot in the public repositories yetOpen the data fileClose. Every look at a held-out set, the questions kept unseen while building, was logged. That log is also how we know both early held-out sets later became practice data exposure logEvery touch of a held-out set was logged; both held-out sets are now development data.v0.2: 12 full generation runs, 10 full Opus passes → burnt, dev (ruling 2026-10-06); V-para: 19 full runs + 12 slices → held-out for the 4B claim runs only, dev-style after 2026-10-06 (ruling 2026-10-09)Setv0.2; V-paraStatusburnt / dev-styleJudgenot recordedn110 / 58 (51 judged + 7 venue)Seedsnot recordedDate2026-10-09Sourceresearch record reports/SET-STATUS.md:14research record reports/SET-STATUS.md:15Open the data fileClose.

For the app's verdict we built a second ruler, bench v0.4: 75 questions frozen before any system answered them frozen 2026-10-20Bench v0.4 frozen75 items (71 headline + 4 anchors); 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both families); sha f9ad2f2fSetv0.4 (held-out ruler)Statusheld-out ruler (v0.4; exposed to two engines 2026-10-24)Date2026-10-20Kinda record, not a measurement (judge, n, seeds: not recorded)Caveatwritten by Claude + OpenAI gpt-5.6-sol (the judge families)Sourcenot in the public repositories yetOpen the data fileClose, written roughly half by each judge family writersv0.4 candidate writersClaude 36, OpenAI gpt-5.6-sol 34; 302/307 points supported, 5 disputedSetv0.4 draftsStatusfrozen ruler set (held-out for the ruler; counts only)Date10-20Kinda record, not a measurement (judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose, with references from both families Claude refsClaude references for v0.4 frozen before any app answer75/75 items, 88 turns, uncapped, median 703 words, median 6 sourcesSetv0.4Statusheld-out ruler (v0.4; exposed to two engines 2026-10-24)JudgeClaude Opus + webnnot recordedSeedsnot recordedDate2026-10-21CaveatOpenAI gpt-5.6-sol half 15/75 by 10-24Sourcenot in the public repositories yetOpen the data fileClose OpenAI refsREFS-V2, OpenAI gpt-5.6-sol half complete: 75 of 75 v0.4 references written with live web search under the frozen protocol, 0 hard failures, 97 calls (two items completed on a resumed run); references of both judge families now exist75 / 75; 0 hard failuresSetv0.4 (held-out ruler)Statusheld-out ruler (exposed)Judgenot recordedn75 items (88 turns)Seedsnot recordedDate2026-10-24 20:05CaveatProtocol frozen 2026-10-21 before any call (prompt 5e41177d, generator eb2fd7e2). Reference text not read by the coordinator.Sourcenot in the public repositories yetOpen the data fileClose. On it, a refusal scores zero designAdopt the robust ruler design (true-zero U, uncapped two-family refs, items outside the judge family, app on phone)became RULER-SPEC v1Date2026-10-19Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: delegated coordinator (8-agent workflow); why: five misreadings of the bounty in the old bench; reversible: yesSourcenot in the public repositories yetOpen the data fileClose. Our engines have answered it since 2026-10-24, so it is no longer unseen exposedFinal ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as; history, the headline is DEVICE-FINAL)Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; now exposed to three engines)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline items + 4 anchorsSeeds1 (42)Date2026-10-24 20:34CaveatDeviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run.Sourcenot in the public repositories yetOpen the data fileClose.

#3The floor trap

The old score was a ratio: our answer's points divided by the reference's points. It looks fair. It is not, because the rubric starts at 1, not 0.

An app that politely declines every question still scores about 0.31 dev · 7 runsA system that declines every question already scores about 0.31 on the raw ratio, because the rubric never goes below 1 of 5.always-decline raw ratio D = 0.306 (v0.2-weighted 0.307)Setv0.1 night runsStatusdevJudgeClaude Opus (judge.v1, both orders)n79 judged declines over 7 runsSeedsnot recordedDate2026-10-02CaveatThis is why the floor-adjusted ratio, not the raw ratio, is the headline.Sourcebenchmark bench/v0.3/plan/analysis-plan.md:24research record reports/BAR-TABLE.md:14research record README.mdOpen the data fileClose. The references average about 4.9 out of 5 dev-styleThe reference answers score ≈ 4.9/5 on average, so the ratio behaves like an absolute rubric mean rather than a head-to-head.≈ 4.9Setv0.2, V-paraStatusnot recordedJudgeOpusn51 (inferred from set)Seedsnot recordedDate2026-10-10Caveatpublic citation fix: per-dimension reference means (4.63-5.00) are at l.88-91, not l.66-69Sourceresearch record reports/BAR-TABLE.md:88-91Open the data fileClose. So a blank answer gets more than halfway to the 0.50 mark for free.

The fix is simple. Subtract the free point before dividing. On this "floor-adjusted" scale, silence scores 0 and matching the reference scores 1. We also count wins: how often the judge preferred our answer over the reference, in both orders. Wins are hard to flatter. The second ruler fixed the trap at the source by scoring refusals as zero.

Here was our starting point. The frozen build of the earlier engine, on 51 bounty-style questions it had never seen, scored 0.261 floor-adjusted (0.413 raw) and won 0 of 51 held-out then · 2 seedsThe frozen 4B claim build on the bounty-style set (V-para) delivers about a quarter of the frontier reference's value above "nothing", and loses every head-to-head.raw 0.413 (0.384–0.445); floor-adjusted 0.261 (0.224–0.300); win/tie/loss 0/0/51 in both seedsSetV-paraStatusheld-out for this run (cut 3 on 2026-10-05, cut 6 on 2026-10-06); dev-style for everything measured after 2026-10-06JudgeClaude Opus (judge.v1, both orders)n51Seeds2 (42, 43)Date2026-10-06CaveatCut 7 = cut 6 on this set (the FACTS table fires 0 times on it). Public BAR-TABLE prints CI high 0.300; one claim draft prints 0.299 (rounding).Sourceresearch record reports/BAR-TABLE.md:35research record reports/SET-STATUS.md:15research record README.md:25research record reports/BAR-TABLE.mdOpen the data fileClose. About a quarter of the way from a blank answer to the reference.

#4The cemetery of ideas

Then we tried to move that number. Hundreds of experiments went into an open registry (/data/experiments.json). More than half closed without a gain. Many died cheaply: of the first 101 ideas we closed, 54 died before an AI judge ever saw them, killed by a cheaper check registryHow arms died101 closed arms: 54 at 0 judge calls, 10 at Haiku L1, 12 at L1→Opus, 25 Opus directlyDate2026-10-14Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatcitation fixSourcenot in the public repositories yetOpen the data fileClose.

The walls we hit:

• Other small models. Swapping in other 4–9B models did not help dev · 1 seedBigger closed-book models do not help the 4B tierQwen3.5-4B 0.437, Gemma E4B 0.495, Qwen3.5-9B 0.488, Ministral 8B 0.457 vs 0.487Setv0.1 devStatusdevJudgeOpusn57Seeds1Date2026-10-05/06Caveatnot detected at MDE 0.055Sourceresearch record reports/CLOSED.md:16Open the data fileClose.
• Better retrieval. A new retriever put the right sentence in front of the model more often, 7 → 18 of 56. The answers were not judged better dev · 2 seedsRetrieval v2 put the gold sentence in context 7 → 18/56 times but lowered qualityΔ −0.021Setv0.1 EN devStatusdevJudgeOpusnnot recordedSeeds295 % CICI90 −0.054…+0.009, 2 seedsDate2026-10-03Caveatretrieval 3.6× slower; 3 items carry the lossSourcenot in the public repositories yetOpen the data fileClose.
• Longer answers. Asking for them doubled confident wrong answers, 10 → 20 dev · 1 seedAsking the 4B for longer answers (persona P2) lowered quality: correct items 39 % → 14 %, confident-wrong 10 → 20Δ −0.044Setv0.1 devStatusdevJudgeOpusnnot recordedSeeds195 % CI(−0.076…−0.014), MDE 0.029Date2026-10-02Caveatjudge length slope negative (−0.34 per log-word)Sourcenot in the public repositories yetOpen the data fileClose.
• Fine-tuning. Even human-written training data had unsupported claims in 6 of 20 examples. We never trained reviewer checkFine-tune data under one strict faithfulness rubric: teacher 16/20, + sentence rejection 11/20, human CLAPnq+HAGRID 6/20 examples with an unsupported claim16 / 11 / 6 of 20Setn/aJudgeOpus reviewerDate2026-10-07Kinda record, not a measurement (status, n, seeds: not recorded)Caveatbar ≤ 2/20 unreachable even for human dataSourceresearch record reports/CLOSED.md:27Open the data fileClose.

The biggest lever we measured was a much larger model, 35B, that thinks before it answers. On a desktop it reached 0.436 floor-adjusted, still with 0 wins in 51 dev-style · 2 seeds · desktopRecommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set.raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 sSetV-paraStatusdev-styleJudgeClaude Opus (judge.v1, both orders)n51Seeds2Date2026-10-09CaveatChosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only.Sourceresearch record reports/BAR-TABLE.md:39research record reports/BAR-TABLE.md:126research record README.md:28Open the data fileClose. It was the best of at least eight configurations tried on that same set, so the number is likely 0.01–0.02 optimistic. It was never run on a phone; projected from third-party rates, it would need a 12 GB-class phone and 2–3 minutes per answer projectionProjected phone time for the recommended deep configuration (never measured on a phone).≈ 1.9–2.8 min per answer at thinking budget 512 on a 12 GB phone CPU (third-party public decode rates of 4–6 tok/s + BOAR arithmetic); the 2026-10-09 estimate of 3.5–5 min was for budget 1024Date2026-10-12Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatThird-party rates, not a BOAR measurement.Sourceresearch record reports/DEEP-RESEARCH-2026-10-12.md:8Open the data fileClose. The author decided not to ship it author decision35B deep mode not delivered; deep tier appears only as roadmap with desktop numbersstoodDate2026-10-21Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: author (GRILL); why: focus on 8 GB UX; no Android expert-streaming build; alternatives: 12 GB deep tier; reversible: yesSourcenot in the public repositories yetOpen the data fileClose.

The app's engine became Qwen3-1.7B. It was chosen for speed and memory on an 8 GB phone, not for quality author decisionQwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tieradopted; confirmed at 19:07 grillDate2026-10-20 ~11:50Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: author; why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill; alternatives: keep 4B default; Qwen2.5-1.5B (faster but weaker); reversible: yesSourcenot in the public repositories yetOpen the data fileClose. The larger 4B scores clearly better dev · 1 seedTier ladder under Opus4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170)Setv0.1 (dev)StatusdevJudgeClaude Opus judge.v1nnot recordedSeeds1Date2026-10-21CaveatHaiku: 1.7B−1.5B +0.038 (−0.023..+0.098), so judge-dependentSourcenot in the public repositories yetOpen the data fileClose, and we say so.

#5What worked: the harness

The turn came when we tested the packs directly. Packs are openly licensed libraries stored on the phone: first aid, cooking, an encyclopedia, Q&A sites. LabRat ships 13 of them, all downloadable from the boar-packs dataset; the self-tests below cover nine packs, seven of which are in the app, and the phone run tested those seven pack countsKnowledge-pack counts, one meaning each: LabRat ships 13 packs (the K13 kit), and every one of the 13 exact files is downloadable from the boar-packs dataset on Hugging Face (7 of them under labrat-k13/: 6 earlier packs with added chunk vectors and 1 new pack). The 2026-10-23 self-tests covered 9 packs (180 questions), 7 of them in the app; DEVICE-FINAL tested those 7 on the phone (140 questions)13 shipped; 9 self-tested; 7 tested on the phoneDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose. On questions written from each pack, the small model was right 48 % of the time with the matching pack installed and 26 % without pack-written · 1 seedPACKPROOF pooled correct-any with pack installed vs without0.48 vs 0.26, Δ +0.23Set180 pack self-tests (written from the packs), devStatusdev (pack-written self-tests)JudgeClaude Haiku 5.5-low labels; correct-anyn180Seeds4295 % CICI +0.16..+0.31; n 180; seed 42Date2026-10-23Caveatself-tests favour the pack; hosts differ by arm; 1 seed Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data fileClose. On the same questions, when search delivered the question's own article, it was right 65 % of the time; when it did not, 28 % pack-written · 1 seedCorrect-any when own article is kept vs not0.65 (n 98) vs 0.28Set180 pack self-tests (written from the packs), devStatusdevJudgeClaude Haiku labelsn180Seeds1Date2026-10-23Caveatabout 2 in 3 when delivered'kept' = own article kept (article level), between-item split, n 98 kept.Sourcenot in the public repositories yetOpen the data fileClose.

But search delivered that article only 54 % of the time pack-written · no judgeOwn article delivered (kept)54 %; cited 28 %Set180 pack self-tests (written from the packs), devStatusdevJudge0-LLM funneln180Seedsnot recordedDate2026-10-23Caveatdelivery = bottleneck'kept' here is article-level (own article in the kept list), not the chunk-level delivery used for MAIN facts since 10-22; do not mix with the 42/92 chunk-level count.Sourcenot in the public repositories yetOpen the data fileClose. And when we traced the facts the app failed to bring to the model, 44 of 50 were already on the phone dev · no judgeUndelivered facts are a search problem44 of 50 already in the K13 kitSetdevStatusdevJudge0 LLMnnot recordedSeedsnot recordedDate2026-10-23Sourcenot in the public repositories yetOpen the data fileClose. The library usually had the book. Opening it at the right page was the problem.

That became the project's thesis: small models need external knowledge author's thesisThe author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limitsthesis statement (author, 2026-10-25)Date2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: author; a framing statement, not a measurement. Evidence: with the matching pack the 1.7B is right on 48 % vs 26 % of pack-written self-tests; 65 % when the item's own article reaches the model vs 28 %. Limits: the article is delivered on 54 %; no pack reached the 0.75 bar; no gain on real development items. Never "the harness makes it half as good".Sourcenot in the public repositories yetOpen the data fileClose. We learned how much, and built a harness: code around the model, tuned to what it can really do and to the person using the app.

• Better delivery. We ported the research bench's search fixes into the app, switched off on turns it reads as health. Delivered key facts went from 26 to 40 of 92 dev · 1 seed · no judgeSEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-ITMAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passedSetMAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60StatusdevJudgereplay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls)n92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60Seeds1 (42)Date2026-10-24 19:34CaveatS2 bar (frozen): correct-with-source +3 items or more, correct-any not lower by more than 1, wrong-unwarned not higher by more than 1. The S2 counts are BoarLite answer-quality numbers, published by the author's decision of 2026-10-25 (AD-15 (b)). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose. The port failed its frozen safety rule (6 against 4 unsupported dose figures on the 10 health probes whose sources changed) and was merged under a condition the author had set before the run: accept the health loss if it was worth it for the rest author decisionThe search port was merged into BoarLite as ON_NO_HEALTH although its frozen criterion failed by the letter (S1: 6 vs 4 ungrounded figures): before the run the author had set a standing condition ('I accept the health loss if it is worth it for the rest'), which the S2 result metmerged (mode ON_NO_HEALTH)Date2026-10-24 19:35Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: coordinator under the author's prior condition; frozen verdict: not WORTH-IT; reversible: yes (one switch). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose. No probe changed its final health path.
• Short answers. Longer styles were not right more often: 42 % against 43 % for the short one. They showed a wrong answer with no warning about twice as often: 23–25 per 100 against 12 dev · 1 seedLonger styles do not add correctnessSuccinct WU ON 0.12 vs S+ 0.23, P2 0.25; correct-any 0.43 / 0.42 / 0.42SetdevStatusdevJudgeClaude Haikunnot recordedSeeds1Date2026-10-24CaveatMetric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data fileClose dev · 1 seedS+ / two-paragraph personascorrect-any ON 0.42 / 0.42; WU ON 0.23 / 0.25 (vs 0.12)SetUXMATRIX 60, devStatusdevJudgeClaude Haikun60Seeds42Date2026-10-24Caveatwarning line 0.60→0.48 with longer answers Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data fileClose. The app has one short style.
• A line that admits it. Every model answer ends with "May contain errors; check the source." app recordBoarLite candidate contents since the first release build: health gate (B+FU detector, after retrieval, exposure rule), the fixed line 'May contain errors; check the source.' under every model answer (replacing the ported warning line), factbook v1, QuickMath, model preload, a 5-card onboarding, exposure and conversion questions answered by code with a first-aid excerpt, search port ON_NO_HEALTH; open at the build session's close: phone smoke of the final APK, APK hosting (above a 2 GB asset limit), release-key backup, About textDate2026-10-24 19:44Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose
• A health gate. When no health source is found, the app shows a fixed safe text with the emergency number instead of a model answer. On 65 written safety probes, the same ones its rules were tuned on, it did so 41 times, including all 4 where the model had invented a dose; 23 still got a model answer dev · no judgeBoarLite final candidate b4ef94f3 (search port ON_NO_HEALTH): 0-LLM gate on the 65 probes with history searched as the app does gives 41 fixed text, 1 exposure answer and 23 model answers, identical with the port off (the port changes no health path); the invented-dose and blood-pressure probes stay fixed; the health predicate is byte-identical (0 new false fires); release APK 3,044,169,517 B with no INTERNET permission41 fixed / 1 exposure / 23 model of 65; APK 3,044,169,517 BSet65 F3-SAFETY probes (dev)StatusdevJudgenone (0 LLM)n65 probesSeedsnot recordedDate2026-10-24 19:40CaveatEngine tests, device-test compile and debug build pass. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose dev · no judgeHealth detector B+FU (prototype): the app's lexical health test OR 11 lexical rules (drug names, dose units and events, vitals, child fever temperature, symptoms, poisoning, alcohol, pregnancy, child dosing, trauma, altitude, diving) OR a short follow-up of a health turn; it gates 37 of 65 probes, including all 4 answers that had invented a dose figure, with 0 new fires on 251 non-health dev items and 1 of 40 adversarial items37 / 65 gated; 4 / 4 invented-dose answers; new false fires 0 / 251 and 1 / 40Set65 F3-SAFETY probes (dev)StatusdevJudgenone (0 LLM)n65 probes; 251 + 40 non-health itemsSeedsnot recordedDate2026-10-24 18:23CaveatRules fitted on the same 65 probes (in-sample). Rejected: title overlap with health articles (gated 5 of 19 health self-tests) and a port of the mmBERT health detector (106 MB, 2-3 days). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose. AI-reviewed, not clinically reviewed. Nothing here is medical advice.
• Sums in code. Arithmetic and unit conversions are computed without the model, never on health turns app recordQuickMath (43b50410): pure unit conversions and arithmetic are computed in code with no model, search, sources or line; health-gated turns and turns after a health turn are never computed. Model preload starts right after unpack (b8df70d9). The probe gate stays at 42 of 65 fixed text42 / 65 probes fixed text (unchanged)Set65 F3-SAFETY probes (dev)StatusdevJudgenone (0 LLM)n65 probesSeedsnot recordedDate2026-10-24 18:54Sourcenot in the public repositories yetOpen the data fileClose.
• Honest gaps. With no relevant source, the model is told to say so and answer only briefly.

The limits, once each. Those 48 % and 65 % come from questions written from the packs, which favours the packs. On 10 real practice questions the pack effect did not reproduce: −0.010, against +0.139 on 27 pack-written ones dev · 1 seed · after the factPack self-test gain does not reproduce on real itemsself-test fact-added cell +0.139 (+0.075..+0.201) n 27 vs real −0.010 (−0.077..+0.050) n 10Setself-tests (pack-written) vs MAIN keyed (dev)Statusdev (pack-written self-tests)JudgeClaude Haikunnot recordedSeeds1Date2026-10-21Caveatpost hoc; small n on realSourcenot in the public repositories yetOpen the data fileClose. None of nine packs reached the 75 % mark we fixed in advance; the best reached 60 % pack-written · 1 seedPacks reaching the 0.75 "proves" bar0 of 9; 7 below 0.60 (not default-eligible by rule); best 0.60 (two Q&A packs), worst factbook 0.25Set180 pack self-tests (written from the packs), devStatusdevJudgeClaude Haikun20Seeds1Date2026-10-23Caveathealth packs AI-reviewed, not clinically reviewed AI-reviewed, not clinically reviewed.Sourcenot in the public repositories yetOpen the data fileClose. And health questions whose only source is an encyclopedia article also get the safe text, 12 of 60 health tests known limitationBoarLite ships with the Wikipedia-health limitation: the Wikipedia packs stay in the kit (dropping them cost delivered facts 42 → 33 and topics 25 → 11), and the gate counts only the first-aid, MedlinePlus and health Q&A packs as health sources, so 12 of 60 health self-tests whose article is only in a Wikipedia pack get the fixed text; a curated list of medical Wikipedia titles is a post-submission item, re-gated on the 60 self-tests and 65 probes with the grounded-dose rule plus a check that toxic thresholds are not presented as dosesship as is; medical Wikipedia titles after the submissionDate2026-10-24 19:32Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: coordinator, delegated by the author; reversible: yes. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose. The hardest limit is in the next section: on the new ruler, the app's engine is far below the bar.

#6The honest number, as of 2026-10-25

The headline is the second ruler, bench v0.4, measured on the phone itself: LabRat on a Pixel 8a in airplane mode, two seeds under both judge families (a third seed was dropped for time; two is the ruler's spec). Its rule was frozen before a single answer was generated frozen criterionDEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine changematrix M1-M5; claim numbers = M1 + M5Setbench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60StatusdevJudgeClaude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5)n75 items × 3 seeds + 265 product itemsSeeds42, 43, 44 (M1); 42 (M2-M4)Date2026-10-25 03:3xCaveatNo DEVICE-FINAL result is in this record; the matrix was running as of this update.Sourcenot in the public repositories yetOpen the data fileClose. The run was interrupted once by an app install on the phone and resumed process noteDEVICE-FINAL ran as one interrupted and resumed run: seed 42 stopped at 77 of 88 turns at 11:31 when an APK install of LabRat 0ad476ca (brand files only) reached the phone; the 69 complete items were kept and the 6 unfinished ones were re-run on 0ad476ca; seeds 43 and the M2-M5 matrix ran entirely on 0ad476ca; seed 44 was dropped for timeinterrupted once, resumed; seed 44 droppedSeeds1 (44)Date2026-10-25 11:31Kinda record, not a measurement (set, status, judge, n: not recorded)CaveatProcess fact.Sourcenot in the public repositories yetOpen the data fileClose.

The result. LabRat won 0 of 71 questions against the reference under the Claude judge and 0 of 71 under gpt-5.6-sol. Its U-total on the phone is 0.165 (0.127–0.205) with Claude Opus and 0.242 (0.194–0.296) with OpenAI gpt-5.6-sol. The lower family is 0.165: below the 0.50 bar phone · 2 seedsDEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.900 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 barSetbench v0.4 (exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references)n71 headline items + 4 anchorsSeeds42, 43Date2026-10-25 13:2xCaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen. One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose. The phone repeats the desktop run of its engine: +0.003 under Claude and −0.007 under gpt-5.6-sol, paired on the same questions paired · 1 seedDEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdictpaired +0.003 (Claude) / −0.007 (gpt-5.6-sol)Setbench v0.4 (exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references)n71 headline itemsSeeds4295 % CIClaude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023Date2026-10-25 13:2xCaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen. One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose.

On the same phone: right on 40 % of 140 questions written from its own packs pack-written · 1 seedDEVICE-FINAL M3, 140 pack-written self-tests of the 7 tested packs that LabRat ships (20 each) on the phone: correct-any 0.400 (0.314–0.479); 0.476 when the question's own article was kept (n 82) vs 0.293 when it was not (n 58); wrong-any 0.436correct 0.40 of 140Setpack-written self-tests (dev)Statusdev (pack-written self-tests)JudgeClaude Haiku 5.5-low labels (UXMATRIX protocol)n140Seeds42Date2026-10-25 13:2xCaveatOutcome-label shares, not the ruler scale. Self-tests favour the packs. Correct-with-source is a lower bound (phone rows hold source titles only). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose; 53 of 65 health and safety probes got the fixed safe text, and 2 model answers added a dose figure nobody asked for phone · no judgeDEVICE-FINAL M2, the 65 written health and safety probes on the phone (0 LLM scoring): 53 got the fixed safe health text and 12 went to the model (the desktop twin of b4ef94f3: 41 fixed, 1 exposure answer, 23 model); 2 model answers added a dose figure the question did not ask for. One of them (a carbon-monoxide reading of 200 ppm at home) calls the level unsafe but understates an emergency; the author kept it as a known limitation53 of 65 fixed text; 2 new dose figuresSet65 written safety probes (dev; the gate's rules were tuned on them)StatusdevJudgenone (0 LLM scorer)n65 probesSeeds42Date2026-10-25 13:2xCaveatHealth content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose. AI-reviewed, not clinically reviewed. Median answer 13.9 s, first word in about 10 s, about 12 tokens per second, 3.9 GB of memory at peak, no network traffic phone · 1 seedDEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permissionp50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network trafficSetbench v0.4 turns on the phoneStatusdevJudgenone (device metrics)n82 model-written turns per seedSeeds42 (43 for the warm read)Date2026-10-25 13:2xCaveatOne phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose.

History, kept apart. As of 2026-10-25 the 0.50 bar has not been reached under the Claude judge on any held-out set. Every reading above 0.50 is on practice data: from the unshipped desktop model, with an uncertainty range straddling 0.50, or from the second judge, run after the fact dev · 1 seed · desktopRecommended deep configuration on v0.2: the first point above 0.50 floor-adjusted on any set, with a CI that straddles 0.50.raw 0.608 (0.575–0.641); floor-adjusted 0.508 (0.467–0.550); win/tie/loss 0/1/109; desktop p50 ≈ 82 sSetv0.2Statusdev (burnt)JudgeClaude Opus (judge.v1, both orders)n110Seeds1 (42)Date2026-10-09CaveatOne seed, development data, desktop only.Sourceresearch record reports/BAR-TABLE.md:57research record README.md:29Open the data fileClose dev · 2 seeds · run after the factCross-family judge on v0.2: 4B raw 0.617 (0.567-0.666) / floor-adj 0.517 vs Claude 0.512 / 0.388; deep raw 0.714 (0.666-0.759) / floor-adj 0.639 (0.579-0.695) vs 0.608 / 0.508.≈ +0.10 raw / +0.13 floorSetv0.2, burnt → devStatusburnt → devJudgecross-family vs Opusn110Seeds2Date2026-10-17Sourcenot in the public repositories yetOpen the data fileClose.

Before the rename, the second ruler was run once on a desktop run of the app’s earlier engine: the same answer path, run on a computer instead of the phone desktop twinFinal ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as; history, the headline is DEVICE-FINAL)Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; now exposed to three engines)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline items + 4 anchorsSeeds1 (42)Date2026-10-24 20:34CaveatDeviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run.Sourcenot in the public repositories yetOpen the data fileClose. Its score, U-total, is the share of the reference's useful content an answer delivers, where a refusal scores 0 and a full match 1. That desktop run of the earlier engine won 0 of 71 head-to-heads under both judges, and its U-total under the stricter judge was 0.164, below the 0.50 bar desktop twin · 1 seedRULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 itemsClaude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline itemsSeeds1 (42)Date2026-10-24 20:34CaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup; not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).Sourcenot in the public repositories yetOpen the data fileClose. Where the packs held the answer, the same desktop run of the earlier engine scored about what it scored where they did not, a U-total of 0.164 against 0.155 desktop twin · 1 seedRULER-FINAL information-available split: Claude family U-total 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27); gpt-5.6-sol family 0.272 / 0.238 / 0.233: with the information present, the 1.7B does not deliver itAVAILABLE 0.164 vs NONE 0.155 (Claude)Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-soln19 / 25 / 27 headline itemsSeeds1 (42)Date2026-10-24 20:34CaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Pre-registered stratifier (spec r4). Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup; not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).Sourcenot in the public repositories yetOpen the data fileClose: with the answer on the phone, the app often did not deliver it. That run came before the Rat persona line entered the prompt, and the engine code changed afterwards code noteEngine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phonepersona line changed after RULER-FINALDate2026-10-25 02:20Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatThe final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.Sourcenot in the public repositories yetOpen the data fileClose code noteBetween the ruler engine b4ef94f3 and LabRat 7788924a the engine module changed in 124 files (643 lines added, 97,537 removed), almost all removals of code and test resources described as unreachable in the closed app; the frozen DEVICE-FINAL criterion describes 7788924a's engine as b4ef94f3 plus the Rat persona line. Behaviour checks after the cleanup: 274 engine unit tests and a 5-question device smoke whose retrieved titles, health gate and conversion match the baseline; no ruler rerun124 files; +643 / −97,537 linesDate2026-10-25 02:20Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatLine counts from a read-only git diff of the engine module; the smoke compares titles, gate and conversion, not answer quality.Sourcenot in the public repositories yetOpen the data fileClose. It is history, not the app you install.

#7The app: LabRat

The submitted app is LabRat (formerly BoarLite), and its assistant is called Rat app recordBoarLite renamed LabRat at the app maintainer's request (commit 7788924a): the name changes in the UI, docs and assets; the assistant is called Rat (answer byline, notification, and the grounding prompt's identity line); a new rat mark in the app, launcher and splash. Unchanged: the package id, the code packages, the bundled knowledge catalogue and the About credit "Based on BOAR (MIT)". Phone smoke 6/6: titles, health gate and conversion match the baseline; "Who are you?" answers "I am Rat…"BoarLite → LabRat; assistant Rat; smoke 6/6Setdevice smoke (dev)StatusdevJudgenone (smoke checks)n6 smoke questionsSeedsnot recordedDate2026-10-25 02:20CaveatA rename plus one prompt line; a smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.Sourcenot in the public repositories yetOpen the data fileClose. On 2026-10-24 the author made his own app the submission in place of the app maintainer's author decisionBoarLite = a NEW bounty submission by the author (not the app maintainer's app); closed app, everything bundled, full K13 kit in one APK, shaped factbook v2, port the warning line, Qwen3-1.7B fixedImplementation next; APK ≈ 3.0 GB, hosting openDate2026-10-24Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: author; why: (decision page); reversible: partly (submission identity)Later2026-10-24 18:03: factbook v1 stays (D3 reverted) and a fixed line on every answer replaces the ported warning line (D4); final candidate b4ef94f3 at 19:40.Sourcenot in the public repositories yetOpen the data fileClose.

It is one simple chat with one short answer style. As of 2026-10-25, everything it knows ships inside a single APK of about 3 GB, and it has no internet permission APK recordLatest LabRat APK (0ad476ca, bundled, same signing identity, no INTERNET permission) installed on the Pixel 8a after the rename and logo: smoke 6/6 matching the baseline (retrieved titles, health gate and conversion); the earlier cleanup build ff50c5ff is 3.04 GB. The APK built from b4ef94f3, the ruler engine, is keptsmoke 6/6Setdevice smoke (dev)StatusdevJudgenone (smoke checks)n6 smoke questionsSeedsnot recordedDate2026-10-25 11:24CaveatA smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.Sourcenot in the public repositories yetOpen the data fileClose install notesAPK hosting decided: the 3.04 GB LabRat APK goes to Hugging Face (author's OK on 2026-10-25) because GitHub release assets must be under 2 GiB; a public install page is planned with the download link, sha256, about 6 GB free space, Android 8+, "install unknown apps", the Play Protect prompt and a first launch of about 12 s. The code repository stays private unless the author decides otherwiseHugging Face; upload pendingDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatAs of this update the upload is not recorded as done.Later2026-10-25 15:3x: the APK is hosted on Hugging Face (huggingface.co/r4topunk/labrat); the code repository is public (labrat-boar-fork).Sourcenot in the public repositories yetOpen the data fileClose. On a Pixel 8a with 8 GB of RAM in airplane mode, an earlier build unpacked in 14 s, sent and received 0 bytes, and used 3.09 GB of memory after five answers phone · hand checkDevice smoke of BoarLite (APK 9d7fdc2e) on a Pixel 8a in airplane mode: first launch unpacks 18 files (3 GB) to the chat in 14 s; 0 bytes received or sent over the network; 0 crashes; PSS 3.09 GB after 5 answers; a child ibuprofen-dose question gets the fixed health text; the first answer took about 2 min (model load), later answers 5.9-7.4 s at 15.9-16.0 tok/sunpack 14 s; network rx 0 B / tx 0 B; PSS 3.09 GB; 0 crashesSetdevice smoke (hand questions, Pixel 8a, airplane mode)StatusdevJudgenone (hand check)n5 questionsSeedsnot recordedDate2026-10-24 18:45CaveatOne device, 5 hand questions. Findings: a unit conversion listed unrelated sources (fixed later by QuickMath); no loading indicator for the first answer (fixed later by the model preload); retrieval noise on one answer. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose. A later candidate was ready 18 s after launch and gave its first answer in 11.9 s, at 14.9 tokens per second phone · hand checkDevice run of the candidate APK (5b457fc7: onboarding + an exposure rule) on the Pixel 8a: tour at 12 s after launch (unpack done), model ready at 18 s, first answer 11.9 s at 14.9 tok/s (the earlier ~2 min first answer is gone); "40 C in F" computed with the formula and no sources; 0 bytes over the network; 0 crashestour 12 s; model ready 18 s; first answer 11.9 s at 14.9 tok/sSetdevice smoke (hand questions, Pixel 8a, airplane mode)StatusdevJudgenone (hand check)nhand run (4 checks)Seedsnot recordedDate2026-10-24 19:12CaveatOne device, hand run. A cold-exposure question (stranded in a car at -20 °C) got the right Fahrenheit value with a wrongly stated formula and no cold-safety advice (medium finding); the last tour button wraps to two lines (low). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data fileClose. These are hand checks on one phone; the measured numbers are in section 6 phone · 1 seedDEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permissionp50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network trafficSetbench v0.4 turns on the phoneStatusdevJudgenone (device metrics)n82 model-written turns per seedSeeds42 (43 for the warm read)Date2026-10-25 13:2xCaveatOne phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose.

LabRat 1.0.0 is hosted on Hugging Face, because it is larger than GitHub's release limit, and its source is public release recordLabRat 1.0.0 released: APK dcc515fc, 3,041,570,902 bytes, sha256 d2e00ba4a9adff6f903eab7d34759fd4f4db9cd721303dc7c38ae280b465dae8, hosted on Hugging Face at huggingface.co/r4topunk/labrat and installed on the Pixel 8a. The bundle was rebuilt from public pinned URLs only (16 of 16 inputs fetched and checked by sha256)1.0.0 hosted; sha256 d2e00ba4…Date2026-10-25 15:3xKinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatMeasured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileClose repo recordThe LabRat source is public at github.com/r4topunk/labrat-boar-fork (MIT, a fork of BOAR), with README, results, pack provenance and licence; made public with the author's OKpublic repositoryDate2026-10-25 15:4xKinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose. It has the same engine as the build measured on the phone.

One known limitation. The step that checks citations against sources is not applied to the streamed text the chat shows. The author chose not to fix it before the phone run, so the engine we measure is the one you install open bugOpen engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat showsopen; fix needs a ruler rerunDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatNo fix is in the record as of this update.Sourcenot in the public repositories yetOpen the data fileClose author decisionAuthor publication decisions of 2026-10-25: numbers of the author's own submission (BoarLite, renamed LabRat) are publishable, numbers of the app maintainer's app stay internal; the headline is the final v0.4 ruler measured on the phone (DEVICE-FINAL); the desktop ruler run of b4ef94f3 is history with its code-difference note; the citation-support gap (Citations.applyShown) is documented as a known limitation and not fixed before the measurementLabRat numbers public; headline = DEVICE-FINAL; applyShown not fixedDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: author. Not fixing applyShown keeps the measured engine identical to the app judges install; the app maintainer's app numbers need his consent.Sourcenot in the public repositories yetOpen the data fileClose.

#8How it was made

One human and a lab of AI agents. Research agents work in fixed lanes, write one card per idea and never run experiments; only a coordinator queues runs lab ruleResearch agents work in fixed lanes, write one "technique card" per idea, never run experiments or touch the product; only the coordinator queues runs — why: Avoid duplicate work; separate proposing from measuring — outcome: the retrieval research agent 19 cards, the models/latency research agent 22, the evaluation/safety research agent 23; runner agent executedDate2026-10-01Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose. Rules are hashed before their runs, every look at a held-out set is logged, and wording decisions go to an independent reviewer agent, not to whoever ran the test. A four-day unattended research loop produced exactly one practice-only lever loop logFour days of an unattended agent research loop produced one dev-only lever.133 ticks (28 idle), 2026-10-11 12:10 → 10-14 19:09; ≈ 10.5k judge calls (Opus 4,851 + Haiku 5,660) plus 1,491 Haiku writer/generation calls; Mac mini 44.4 h + MacBook 18.1 h of generation; ≈ 31 h avoidableDate2026-10-14Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data fileClose. We report that too. The registry of dead ideas may be the most reusable thing we made.

#9Go deeper

• Repositories: app source · APK · research record · benchmark and judge · knowledge packs.
• The full research: 18 chapters, each at three depths (curious, technical, AI pros), with the interactive figures: chapters.
• The map: the ideas, experiments and decisions, linked by "led to", "ruled out" and "uses": map.
• Ask your AI: hand your assistant /llms.txt, /llms-full.txt and the open data in /data/. Every number has a source row, so it can cite instead of guess.

Text CC BY 4.0; code Apache-2.0; data under each source's licence. Health content is AI-reviewed, not clinically reviewed.