The research · Act V, Where it stands · Chapter 16
Where it stands (live)
As of 2026-10-25 03:30 the research lines are closed and the submitted app (LabRat, formerly BoarLite) runs offline on a phone.
Full thesis
Its headline number is the final ruler (bench v0.4) measured on the phone: 0 of 71 wins and a lower-family U-total of 0.165 on the phone, below the 0.50 bar. As of 2026-10-25, the record also holds history: the same ruler on the desktop run of the earlier engine (one seed, before the Rat persona line) won 0 of 71 under both judge families with a lower-family U-total of 0.164, below the 0.50 bar; under the Claude judge the old ratio did not reach 0.50 on the earlier held-out sets; and no unexposed held-out number exists. The thesis the record presents is the author's: small models need external knowledge, and the app is a harness built around what a small model can really do The author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limitsthesis statement (author, 2026-10-25)Date2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: author; a framing statement, not a measurement. Evidence: with the matching pack the 1.7B is right on 48 % vs 26 % of pack-written self-tests; 65 % when the item's own article reaches the model vs 28 %. Limits: the article is delivered on 54 %; no pack reached the 0.75 bar; no gain on real development items. Never "the harness makes it half as good".Sourcenot in the public repositories yetOpen the data file; the outcome labels of the closing tests are its evidence and its limits. Three metric families stay separate. This chapter is finalized at closure (v1.0).
Where it stands · three kinds of number, never on one scale
A · The old ruler
historyfloor-adjusted ratio vs a frontier model with web · 51 bounty-style questions
reports/BAR-TABLE.md:35research record reports/SET-STATUS.md:15research record README.md:25research record reports/BAR-TABLE.mdOpen the data fileRecommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set.raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 sSetV-paraStatusdev-styleJudgeClaude Opus (judge.v1, both orders)n51Seeds2Date2026-10-09CaveatChosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only.Sourceresearch record reports/BAR-TABLE.md:39research record reports/BAR-TABLE.md:126research record README.md:28Open the data fileAnchor: a small frontier model with no web access (Claude Haiku 5.5) on the bounty-style set.raw 0.588 (0.559–0.615); floor-adjusted 0.48 (CI unknown)SetV-paraStatusheld-out at the time (2026-10-05)JudgeClaude Opus (judge.v1, both orders)n51Seeds1Date2026-10-05CaveatThe best deep configuration (0.436, CI high 0.472) does not reach this anchor either.Sourceresearch record reports/BAR-TABLE.md:44Open the data fileDot = mean of two seeds, line = 95 % interval. Only the first row is phone-sized; the deep rows ran on a desktop.Data table
| configuration | floor-adjusted (95 % CI) | raw (95 % CI) | wins |
|---|---|---|---|
| 4B claim build (phone-sized model) | 0.261 (0.224–0.300) | 0.413 (0.384–0.445) | 0 / 51 |
| Deep tier, 35B (no thinking) | 0.329 (0.294–0.362) | 0.467 (0.439–0.494) | 0 / 51 |
| + thinking, 256 (tokens, deep tier) | 0.387 (0.349–0.426) | 0.514 (0.483–0.544) | 0 / 51 |
| + thinking, 1024 (tokens, deep tier) | 0.422 (0.385–0.459) | 0.542 (0.513–0.571) | 0 / 51 |
| + thinking, 512 (tokens, recommended) | 0.436 (0.401–0.472) | 0.553 (0.524–0.581) | 0 / 51 |
| Claude Haiku, no web (anchor, not ours) | 0.480 (CI not computed) | 0.588 (0.559–0.615) | — |
B · The closing tests
pack-writtenpercent of answers right · 180 questions written from the packs · no 0.50 line here
C · The new ruler
on the phoneshare of the reference's useful content · 71 head-to-heads · two judge families
0 of 71 wins under both judge families
Pixel 8a, airplane mode, two seeds. The stricter judge sets the verdict: below the 0.50 bar.
History, not the app you install
The desktop run of the earlier engine, before the Rat persona line, had a U-total of 0.164 under the stricter judge and won 0 of 71. The phone repeated it.
Not shown, as of 25 October
As of 25 October 2026, the research is closed and the bar has not been shown to be reached. The number that counts is the new exam measured on the phone: 0 of 71 wins and a lower-family U-total of 0.165 on the phone, below the 0.50 bar. The same exam on a desktop copy of the app's earlier engine won none of 71 head-to-heads and stayed below the bar. On the earlier sealed bounty-style questions the frozen 4B build delivered about a quarter of the frontier answer's value above nothing (0.261) and won none of 51. RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 itemsClaude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline itemsSeeds1 (42)Date2026-10-24 20:34CaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup; not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).Sourcenot in the public repositories yetOpen the data file
Three numbers that are not the same kind
You will see three kinds of number on this site: the old ratio against the frontier answer, answer labels from the closing tests ("right", "wrong without a warning"), and the new exam's score (on the phone and on a desktop copy: below the bar). We never put them on one scale.
Our claim, in our words
The prepared sentence: "BOAR is not shown to be half as good as frontier AI with web." It was written for the research engine; it will be revised for LabRat (formerly BoarLite) and posted with a link to this article, and the change log will say so.
The numbers that drive this chapter
- Panel A, judge.v1 ratio (old scale; history since 2026-10-25,
history.old_scale). V-para (held-out at run time, Opus, 2 seeds): the frozen 4B build raw 0.413 (0.384–0.445), floor-adjusted 0.261 (0.224–0.300), 0 of 51 wins The frozen 4B claim build on the bounty-style set (V-para) delivers about a quarter of the frontier reference's value above "nothing", and loses every head-to-head.raw 0.413 (0.384–0.445); floor-adjusted 0.261 (0.224–0.300); win/tie/loss 0/0/51 in both seedsSetV-paraStatusheld-out for this run (cut 3 on 2026-10-05, cut 6 on 2026-10-06); dev-style for everything measured after 2026-10-06JudgeClaude Opus (judge.v1, both orders)n51Seeds2 (42, 43)Date2026-10-06CaveatCut 7 = cut 6 on this set (the FACTS table fires 0 times on it). Public BAR-TABLE prints CI high 0.300; one claim draft prints 0.299 (rounding).Sourceresearch recordreports/BAR-TABLE.md:35research recordreports/SET-STATUS.md:15research recordREADME.md:25research recordreports/BAR-TABLE.mdOpen the data file. v0.2 (burnt → dev): raw 0.512 / floor-adjusted 0.388, 0/0/110 and 0/1/109 The same 4B claim build on the second set (v0.2, later burnt to dev).raw 0.512 (0.478–0.549); floor-adjusted 0.388 (0.346–0.434); win/tie/loss 0/0/110 and 0/1/109 (per seed)Setv0.2Statusheld-out at run time; burnt → dev by ruling 2026-10-06JudgeClaude Opus (judge.v1, both orders)n110Seeds2Date2026-10-06CaveatClaim draft v4.2.4 (§0 and Appendix A) carries the cut-3 values 0.513 / 0.389; the cut-6 = cut-7 values here are the ones to publish.Sourceresearch recordreports/BAR-TABLE.md:53research recordREADME.md:26Open the data file. Bar reading: no configuration reaches floor-adjusted 0.50; the formal raw rule is met only by the desktop deep tier on development data Bar status: no configuration reaches 0.50 on the primary (floor-adjusted) reading; the formal raw rule is met only on development data by the deep tier with thinking.best floor-adjusted on V-para = 0.436 (CI high 0.472), short by 0.064; formal raw rule (CI low > 0.50) met by deep + thinking on dev data only (desktop; the v0.2 leg of the 512 budget is 1 seed); 0 wins on every Claude-judged row of V-para and v0.2SetV-para; v0.2Statusdev / dev-styleJudgeClaude Opus (judge.v1, both orders)n51 / 110Seedssee rowsDate2026-10-10CaveatNever present as "bar reached". Under the Claude judge on held-out data the bar was not reached.Sourceresearch recordreports/BAR-TABLE.md:80research recordREADME.md:42research recordreports/BAR-TABLE.md:73,80research recordreports/BAR-TABLE.md:66-80Open the data file. Closest configuration (desktop, dev-style): floor-adjusted 0.436 (0.401–0.472), 0 of 51 Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set.raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 sSetV-paraStatusdev-styleJudgeClaude Opus (judge.v1, both orders)n51Seeds2Date2026-10-09CaveatChosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only.Sourceresearch recordreports/BAR-TABLE.md:39research recordreports/BAR-TABLE.md:126research recordREADME.md:28Open the data file; never delivered 35B deep mode not delivered; deep tier appears only as roadmap with desktop numbersstoodDate2026-10-21Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: author (GRILL); why: focus on 8 GB UX; no Android expert-streaming build; alternatives: 12 GB deep tier; reversible: yesSourcenot in the public repositories yetOpen the data file. Cross-family (post-hoc): V-para 0.361 / 0.577 floor-adjusted, win 0.00–0.01 A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate.4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01SetV-paraStatusdev-styleJudgecross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered)n51 (408 judge calls)Seeds2Date2026-10-16CaveatPost-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline.Sourcenot in the public repositories yetOpen the data file; v0.2 The cross-family judge on v0.2.4B: raw 0.617 (0.567–0.666), floor-adjusted 0.517 (2 seeds); deep + think 512: raw 0.714 (0.666–0.759), floor-adjusted 0.639 (0.579–0.695) (1 seed); deep − 4B +0.097 vs +0.096 under Claude; Spearman 0.90–0.97; win-rate 2.7 % / 6.4 % vs 0 % under ClaudeSetv0.2Statusdev (burnt)Judgecross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered)n110 (660 judge calls)Seeds4B 2, deep 1Date2026-10-17CaveatPost-hoc. The Claude judge is the stricter family by ≈ 0.10 on both sets; the offset is stable across sets and arms.Sourcenot in the public repositories yetOpen the data file. Set status: both former held-out sets are now development data Every touch of a held-out set was logged; both held-out sets are now development data.v0.2: 12 full generation runs, 10 full Opus passes → burnt, dev (ruling 2026-10-06); V-para: 19 full runs + 12 slices → held-out for the 4B claim runs only, dev-style after 2026-10-06 (ruling 2026-10-09)Setv0.2; V-paraStatusburnt / dev-styleJudgenot recordedn110 / 58 (51 judged + 7 venue)Seedsnot recordedDate2026-10-09Sourceresearch recordreports/SET-STATUS.md:14research recordreports/SET-STATUS.md:15Open the data file. - Panel B, outcome labels (closing tests; dev; pack-written where named; Haiku labels). With the pack, correct-any 0.48 vs 0.26 without PACKPROOF pooled correct-any with pack installed vs without0.48 vs 0.26, Δ +0.23Set180 pack self-tests (written from the packs), devStatusdev (pack-written self-tests)JudgeClaude Haiku 5.5-low labels; correct-anyn180Seeds4295 % CICI +0.16..+0.31; n 180; seed 42Date2026-10-23Caveatself-tests favour the pack; hosts differ by arm; 1 seed Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data file; 0 of 9 packs at 0.75 Packs reaching the 0.75 "proves" bar0 of 9; 7 below 0.60 (not default-eligible by rule); best 0.60 (two Q&A packs), worst factbook 0.25Set180 pack self-tests (written from the packs), devStatusdevJudgeClaude Haikun20Seedsnot recordedDate2026-10-23Caveathealth packs AI-reviewed, not clinically reviewed AI-reviewed, not clinically reviewed.Sourcenot in the public repositories yetOpen the data file; 44 of 50 undelivered facts already in the kit Undelivered facts are a search problem44 of 50 already in the K13 kitSetdevStatusdevJudge0 LLMnnot recordedSeedsnot recordedDate2026-10-23Sourcenot in the public repositories yetOpen the data file; the shipped style on the engine release candidate: correct-any 0.43, wrong-unwarned 0.12 Succinct on ENGINE-RC (page numbers)ON CWS 0.30, correct-any 0.43, WU 0.12; OFF correct-any 0.22, WU 0.10SetUXMATRIX 60, devStatusdevJudgeClaude Haiku labelsn60Seeds42Date2026-10-24CaveatPASS vs frozen CWS ≥0.28, WU OFF ≤0.10 Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data file.
- Panel C, the second ruler. Bench v0.4 frozen Bench v0.4 frozen75 items (71 headline + 4 anchors); 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both families); sha f9ad2f2fSetv0.4 (held-out ruler)Statusheld-out ruler (v0.4; exposed to two engines 2026-10-24)Date2026-10-20Kinda record, not a measurement (judge, n, seeds: not recorded)Caveatwritten by Claude + OpenAI gpt-5.6-sol (the judge families)Sourcenot in the public repositories yetOpen the data file; references of both families 75/75 Claude references for v0.4 frozen before any app answer75/75 items, 88 turns, uncapped, median 703 words, median 6 sourcesSetv0.4Statusheld-out ruler (v0.4; exposed to two engines 2026-10-24)JudgeClaude Opus + webnnot recordedSeedsnot recordedDate2026-10-21CaveatOpenAI gpt-5.6-sol half 15/75 by 10-24Sourcenot in the public repositories yetOpen the data file. On the phone (headline): the final on-phone run landed (criterion frozen 2026-10-25) DEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine changematrix M1-M5; claim numbers = M1 + M5Setbench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60StatusdevJudgeClaude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5)n75 items × 3 seeds + 265 product itemsSeeds42, 43, 44 (M1); 42 (M2-M4)Date2026-10-25 03:3xCaveatNo DEVICE-FINAL result is in this record; the matrix was running as of this update.Sourcenot in the public repositories yetOpen the data file; when it lands, its M1 (win-rate first, both families, seeds stated, U-total per family, the lower family against the bar) and M5 (speed, memory, offline) fill the headline slot in whichever direction they fall; until then the panel predicts nothing. On the desktop (history,
history.desktop_ruler): BoarLite’s earlier engine, one seed: 0 of 71 wins under both families; U-total Claude 0.164 (0.123–0.208), gpt-5.6-sol 0.245 (0.194–0.300); lower family 0.164 (0.123–0.206), below the 0.50 bar; information-available split as in chapter 12 RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 itemsClaude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline itemsSeeds1 (42)Date2026-10-24 20:34CaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup; not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).Sourcenot in the public repositories yetOpen the data file; labelled "desktop twin, one seed, not a phone run" and "earlier engine, before the Rat persona line". - Product state. Main engine Qwen3-1.7B Qwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tieradopted; confirmed at 19:07 grillDate2026-10-20 ~11:50Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: author; why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill; alternatives: keep 4B default; Qwen2.5-1.5B (faster but weaker); reversible: yesSourcenot in the public repositories yetOpen the data file; the 4B leads it by +0.109 under Opus on v0.1 51 dev Tier ladder under Opus4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170)Setv0.1 (dev)StatusdevJudgeClaude Opus judge.v1nnot recordedSeeds1Date2026-10-21CaveatHaiku: 1.7B−1.5B +0.038 (−0.023..+0.098), so judge-dependentSourcenot in the public repositories yetOpen the data file; submission LabRat (formerly BoarLite; renamed 2026-10-25) BoarLite renamed LabRat at the app maintainer's request (commit 7788924a): the name changes in the UI, docs and assets; the assistant is called Rat (answer byline, notification, and the grounding prompt's identity line); a new rat mark in the app, launcher and splash. Unchanged: the package id, the code packages, the bundled knowledge catalogue and the About credit "Based on BOAR (MIT)". Phone smoke 6/6: titles, health gate and conversion match the baseline; "Who are you?" answers "I am Rat…"BoarLite → LabRat; assistant Rat; smoke 6/6Setdevice smoke (dev)StatusdevJudgenone (smoke checks)n6 smoke questionsSeedsnot recordedDate2026-10-25 02:20CaveatA rename plus one prompt line; a smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.Sourcenot in the public repositories yetOpen the data file, built from final candidate BoarLite final candidate b4ef94f3 (search port ON_NO_HEALTH): 0-LLM gate on the 65 probes with history searched as the app does gives 41 fixed text, 1 exposure answer and 23 model answers, identical with the port off (the port changes no health path); the invented-dose and blood-pressure probes stay fixed; the health predicate is byte-identical (0 new false fires); release APK 3,044,169,517 B with no INTERNET permission41 fixed / 1 exposure / 23 model of 65; APK 3,044,169,517 BSet65 F3-SAFETY probes (dev)StatusdevJudgenone (0 LLM)n65 probesSeedsnot recordedDate2026-10-24 19:40CaveatEngine tests, device-test compile and debug build pass. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file plus the Rat persona line and removed code Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phonepersona line changed after RULER-FINALDate2026-10-25 02:20Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatThe final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.Sourcenot in the public repositories yetOpen the data file, offline on a Pixel 8a with the first answer in 11.9 s on a pre-rename APK Device run of the candidate APK (5b457fc7: onboarding + an exposure rule) on the Pixel 8a: tour at 12 s after launch (unpack done), model ready at 18 s, first answer 11.9 s at 14.9 tok/s (the earlier ~2 min first answer is gone); "40 C in F" computed with the formula and no sources; 0 bytes over the network; 0 crashestour 12 s; model ready 18 s; first answer 11.9 s at 14.9 tok/sSetdevice smoke (hand questions, Pixel 8a, airplane mode)StatusdevJudgenone (hand check)nhand run (4 checks)Seedsnot recordedDate2026-10-24 19:12CaveatOne device, hand run. A cold-exposure question (stranded in a car at -20 °C) got the right Fahrenheit value with a wrongly stated formula and no cold-safety advice (medium finding); the last tour button wraps to two lines (low). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file; search port merged as the search port (off on health turns) against its frozen verdict, by the author's prior condition SEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-ITMAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passedSetMAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60StatusdevJudgereplay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls)n92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60Seeds1 (42)Date2026-10-24 19:34CaveatS2 bar (frozen): correct-with-source +3 items or more, correct-any not lower by more than 1, wrong-unwarned not higher by more than 1. The S2 counts are BoarLite answer-quality numbers, published by the author's decision of 2026-10-25 (AD-15 (b)). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file; ships with the Wikipedia-health limitation BoarLite ships with the Wikipedia-health limitation: the Wikipedia packs stay in the kit (dropping them cost delivered facts 42 → 33 and topics 25 → 11), and the gate counts only the first-aid, MedlinePlus and health Q&A packs as health sources, so 12 of 60 health self-tests whose article is only in a Wikipedia pack get the fixed text; a curated list of medical Wikipedia titles is a post-submission item, re-gated on the 60 self-tests and 65 probes with the grounded-dose rule plus a check that toxic thresholds are not presented as dosesship as is; medical Wikipedia titles after the submissionDate2026-10-24 19:32Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: coordinator, delegated by the author; reversible: yes. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file.
- Claim text: the prepared sentence is "BOAR is not shown to be half as good as frontier AI with web" (draft v4.2.4) Claim text sentence (draft v4.2.4, as of 2026-10-17)"BOAR is not shown to be half as good as frontier AI with web."Date2026-10-17Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatdraft, not published at time of readingSourcenot in the public repositories yetOpen the data file. Whether it was posted is not in the record; it must be revised for LabRat before closure (“Claim text on the bounty”); by the author's decision of 2026-10-25 the revised claim is posted after v1.0 with a link to the article. The plan (2026-10-24 PM): lead with what the app does and the final ruler numbers labelled with both judge families, seeds, desktop twin or phone, and win-rate, with product numbers labelled by engine; wording to an independent reviewer, then the author Claim-text plan (2026-10-24 PM hand-off): lead with what the app does and the final ruler numbers labelled with both judge families, the seed count, "desktop twin" and the win-rate, plus measured product numbers each labelled with the engine they come from; wording goes to an independent reviewer, then the author. Also pending: a decisions-and-evidence page for the judges, a short demo post and the app's About textplan; not postedDate2026-10-24 20:5xKinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatThe ruler numbers stay internal until the author rules on AD-15; this row states the plan, not the numbers.Sourcenot in the public repositories yetOpen the data file.
6.1 The frozen 4B build on held-out sets (standing as of 2026-10-24; unchanged since the 2026-10-05/06 claim runs)
On V-para (held-out at run time; n = 51; seeds 42 and 43; Opus) the 4B claim build scores raw 0.413 (0.384–0.445), floor-adjusted 0.261 (0.224–0.300), above-decline 0.155 (0.112–0.200), and loses all 51 comparisons in both seeds. On v0.2 (held-out at run time, since burnt; n = 110; two seeds) it scores raw 0.512 (0.478–0.549), floor-adjusted 0.388 (0.346–0.434), W/T/L 0/0/110 and 0/1/109. Across the seven builds, held-out raw moved from 0.506 (baseline) to 0.512 (cut 6), +0.006, within noise. The cut 6 vs cut 3 comparison on v0.2 (Δ −0.0009, −0.0095…+0.0077) reuses 323 of 334 verdicts for byte-identical answers and is not an independent re-judging. For reference, Claude Haiku 5.5 without web scores raw 0.588 (0.559–0.615), floor-adjusted 0.48 (CI not computed) on V-para (1 run, 2026-10-05).
6.10 Small models need external knowledge: what the closing tests support
The author closed the research lines on 2026-10-23 with the hypothesis "right whenever the content is in a pack"; on 2026-10-25 he stated the thesis as a design statement: small models need external knowledge, and the project built a harness around a small model's real capabilities, tuned for the app user's experience The author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limitsthesis statement (author, 2026-10-25)Date2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: author; a framing statement, not a measurement. Evidence: with the matching pack the 1.7B is right on 48 % vs 26 % of pack-written self-tests; 65 % when the item's own article reaches the model vs 28 %. Limits: the article is delivered on 54 %; no pack reached the 0.75 bar; no gain on real development items. Never "the harness makes it half as good".Sourcenot in the public repositories yetOpen the data file. The numbers below are its evidence and its limits; they do not establish a quality level. Closing tests ran on the desktop twin with outcome labels (Claude Haiku 5.5-low; §2.13 of the claims page): on 180 pack-written self-tests of 9 packs (1.7B Succinct, seed 42), correct-any is 0.48 with the pack vs 0.26 without (+0.23, +0.16…+0.31); 0.65 when the item's own article is kept (n 98) vs 0.28; the own article is kept on 54 % and cited on 28 % (article level); 0 of 9 packs reach the 0.75 bar fixed in advance PACKPROOF pooled correct-any with pack installed vs without0.48 vs 0.26, Δ +0.23Set180 pack self-tests (written from the packs), devStatusdev (pack-written self-tests)JudgeClaude Haiku 5.5-low labels; correct-anyn180Seeds4295 % CICI +0.16..+0.31; n 180; seed 42Date2026-10-23Caveatself-tests favour the pack; hosts differ by arm; 1 seed Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data file. The hypothesis therefore holds only in the conditional form, and on real development items the pack gain does not transfer (fact-added cell −0.010, −0.077…+0.050, n 10, vs +0.139 on self-tests) Pack self-test gain does not reproduce on real itemsself-test fact-added cell +0.139 (+0.075..+0.201) n 27 vs real −0.010 (−0.077..+0.050) n 10Setself-tests (pack-written) vs MAIN keyed (dev)Statusdev (pack-written self-tests)JudgeClaude Haikunnot recordedSeeds1Date2026-10-21Caveatpost hoc; small n on realSourcenot in the public repositories yetOpen the data file. Pack shaping (factbook) doubled delivery and correct answers on 20 self-tests and moved Opus U 0.316 → 0.436 (+0.121, +0.030…+0.221), NOT-CONFIRMED by one new warned confident error Shaped factbookfact chunk kept 7→15/20; correct 5→10/20; Opus U 0.316→0.436 (+0.121, +0.030..+0.221)Setfactbook self-tests (pack-written)Statusdev (pack-written self-tests)JudgeClaude Haiku; Claude Opusnnot recordedSeeds2Date2026-10-23CaveatNOT-CONFIRMED (1 new warned CE); does not transfer to the native engineSourcenot in the public repositories yetOpen the data file. The closed engine passed every 0-LLM validation suite and cut data read per question 125.09 → 21.26 MB ENGINE-RC passes the 0-LLM validation suitesidentity 97/97 & 180/180; MAIN delivered 42; ST fact 84 vs 77; routes 103/110; 21.26 MB/qSetdevStatusdevJudgereplaynnot recordedSeedsnot recordedDate2026-10-23CaveatV1 letter-FAIL pre-existingSourcenot in the public repositories yetOpen the data file. The 2026-10-24 validation judge kept Succinct: correct-any 0.43 vs 0.42 for longer styles, wrong-unwarned 0.12 vs 0.23–0.25 (UXMATRIX 60, seed 42) Succinct on ENGINE-RC (page numbers)ON CWS 0.30, correct-any 0.43, WU 0.12; OFF correct-any 0.22, WU 0.10SetUXMATRIX 60, devStatusdevJudgeClaude Haiku labelsn60Seeds42Date2026-10-24CaveatPASS vs frozen CWS ≥0.28, WU OFF ≤0.10 Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.Sourcenot in the public repositories yetOpen the data file; the Succinct reading moved from 0.35 on 10-23 to 0.43 on 10-24 on the same items after engine changes, with no isolated cause. Subject carry restored the turn-1 source on follow-ups (0.083 → 0.708) without changing accuracy (0.458) Follow-ups on ENGINE-RC, with and without subject carrycorrect-any 0.458 both; on-referent 1.00; turn-1 source 0.083→0.708; WU 2→4 of 48SetF8 devStatusdevJudgeClaude Haikun48Seeds42Date2026-10-24Caveathealth follow-ups 0.19 both Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. AI-reviewed, not clinically reviewed.Sourcenot in the public repositories yetOpen the data file.
6.11 From the maintainer's app to BoarLite, renamed LabRat
The research engine was never the submitted system. Until 2026-10-24 the submission was the app maintainer's native app; its release candidate was scored internally and those numbers, like the desktop comparison of the native engine with ENGINE-RC, the first v0.4 answers and a probe check of its current release, stay withheld: by the author's decision of 2026-10-25 the app maintainer's numbers stay internal. On 2026-10-24 the author declared BoarLite, his own closed app (Qwen3-1.7B fixed, Succinct only, the K13 kit with factbook v1 in one 3.04 GB APK, no INTERNET permission, English only), the submission (amended by). By 19:40 its final candidate b4ef94f3 added a lexical health gate (B+FU; 41 of 65 dev safety probes end in fixed safe text, all 4 previously invented doses among them; rules fitted on those probes) Health detector B+FU (prototype): the app's lexical health test OR 11 lexical rules (drug names, dose units and events, vitals, child fever temperature, symptoms, poisoning, alcohol, pregnancy, child dosing, trauma, altitude, diving) OR a short follow-up of a health turn; it gates 37 of 65 probes, including all 4 answers that had invented a dose figure, with 0 new fires on 251 non-health dev items and 1 of 40 adversarial items37 / 65 gated; 4 / 4 invented-dose answers; new false fires 0 / 251 and 1 / 40Set65 F3-SAFETY probes (dev)StatusdevJudgenone (0 LLM)n65 probes; 251 + 40 non-health itemsSeedsnot recordedDate2026-10-24 18:23CaveatRules fitted on the same 65 probes (in-sample). Rejected: title overlap with health articles (gated 5 of 19 health self-tests) and a port of the mmBERT health detector (106 MB, 2-3 days). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file, a fixed "may contain errors" line, code-computed conversions (QuickMath) and the bench search ported as ON_NO_HEALTH. On a Pixel 8a in airplane mode it unpacks in 14 s, sends 0 bytes, holds PSS 3.09 GB and gives a first answer in 11.9 s at 14.9 tok/s (one device, hand checks) Device smoke of BoarLite (APK 9d7fdc2e) on a Pixel 8a in airplane mode: first launch unpacks 18 files (3 GB) to the chat in 14 s; 0 bytes received or sent over the network; 0 crashes; PSS 3.09 GB after 5 answers; a child ibuprofen-dose question gets the fixed health text; the first answer took about 2 min (model load), later answers 5.9-7.4 s at 15.9-16.0 tok/sunpack 14 s; network rx 0 B / tx 0 B; PSS 3.09 GB; 0 crashesSetdevice smoke (hand questions, Pixel 8a, airplane mode)StatusdevJudgenone (hand check)n5 questionsSeedsnot recordedDate2026-10-24 18:45CaveatOne device, 5 hand questions. Findings: a unit conversion listed unrelated sources (fixed later by QuickMath); no loading indicator for the first answer (fixed later by the model preload); retrieval noise on one answer. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file.
A pre-registered criterion failed by the letter and overridden. The search port delivers more facts (MAIN keyed 26 → 40, chunk level, 0-LLM replay) but failed every frozen safety check: P1 gate d (3 new dose figures), then the grounded-dose re-gate (ungrounded figures 9 vs 5), then ON_NO_HEALTH's (6 vs 4; no probe changes final path), while ON_NO_HEALTH passed its usefulness screen (on the 51 changed UXMATRIX items, correct-with-source 9 → 15, correct-any 20 → 21, wrong-unwarned 31 → 30; Haiku labels, seed 42) SEARCHPORT Addendum 2, S2 counts: on the 51 UXMATRIX items whose kept list changed, correct-with-source 9 → 15, correct-any 20 → 21, wrong-unwarned 31 → 30 (port off → ON_NO_HEALTH)CWS 9 → 15; correct-any 20 → 21; WU 31 → 30 (of 51)SetUXMATRIX 60 (dev)StatusdevJudgeClaude Haikun51 changed items of UXMATRIX 60Seeds1 (42)Date2026-10-24 19:35CaveatMetric: outcome-label counts (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio, not the ruler U-total. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission.Sourcenot in the public repositories yetOpen the data file. It was merged because the author had stated before the run that he would accept a health loss if the rest gained SEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-ITMAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passedSetMAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60StatusdevJudgereplay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls)n92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60Seeds1 (42)Date2026-10-24 19:34CaveatS2 bar (frozen): correct-with-source +3 items or more, correct-any not lower by more than 1, wrong-unwarned not higher by more than 1. The S2 counts are BoarLite answer-quality numbers, published by the author's decision of 2026-10-25 (AD-15 (b)). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file. We report this as an adoption against a frozen verdict, not as a pass.
Known limits: health sources exclude Wikipedia, so 12 of 60 health self-tests with only a Wikipedia article get the fixed text (shipped); the chat path does not apply the pipeline's citation-support cleaning Finding from the final ruler run: BoarLite's chat shows the streamed model text, so the pipeline's cleaned answer (the citation support check CT-1 and attribution) is not used on the plain path; the run scored what the chat shows. Logged as a possible BoarLite bugDate2026-10-24 20:34Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Sourcenot in the public repositories yetOpen the data file, a known limitation the author chose not to fix before the measurement so that the measured engine is the installed one Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat showsopen; fix needs a ruler rerunDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatNo fix is in the record as of this update.Sourcenot in the public repositories yetOpen the data file. The final ruler run on b4ef94f3 (desktop twin, one seed, both families) Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as; history, the headline is DEVICE-FINAL)Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; now exposed to three engines)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline items + 4 anchorsSeeds1 (42)Date2026-10-24 20:34CaveatDeviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run.Sourcenot in the public repositories yetOpen the data file is reported in §3.6 as history. No measurement in §6.1–§6.10 is a measurement of BoarLite.
After the ruler run (2026-10-24 21:30 → 2026-10-25 11:24). UI review, then a cleanup round of owner decisions: the Expo/React Native tree, Detailed, adaptive routing, persona and length options, Deep Research beyond the recap, the long-chat summary (on the phone the 1.7B used only the last 3 exchanges), the PIN lock and Portuguese removed; Settings simplified; licence texts in About BoarLite screens review by the author (2026-10-24 evening, UI only, engine untouched so the ruler run on b4ef94f3 still applies): a calmer first tour card, Knowledge pack icons, time and tok/s shown above the sources, no Free memory switch (always on), no PIN lock (code removed), haptics on by default, a new About page, reset texts without PIN, the chat top bar showing only the name, a drawer brand card counting the 13 knowledge packs, and the mark in the accent colour. Phone smoke of the resulting APK (b45f3aa8) on the Pixel 8a: a cited answer in 24 s, the child-ibuprofen question gets the fixed health text, 40 °C = 104 °F computed, Large text checked on all main screensUI only; smoke on the Pixel 8a passedSetdevice smoke (dev)StatusdevJudgenone (hand checks)nhand smokeSeedsnot recordedDate2026-10-24 23:48CaveatA smoke test, not a quality measurement. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.Sourcenot in the public repositories yetOpen the data file. On 2026-10-25 the app was renamed LabRat at the app maintainer's request, its assistant Rat (7788924a, same package id). This is an engine change after the ruler: the grounding prompt's identity line now reads "You are Rat, the assistant in LabRat…" (was "You are BOAR…"), and the engine module changed in 124 files, almost all removals of code described as unreachable, checked by 274 unit tests and a baseline-matching device smoke, not by a ruler rerun Between the ruler engine b4ef94f3 and LabRat 7788924a the engine module changed in 124 files (643 lines added, 97,537 removed), almost all removals of code and test resources described as unreachable in the closed app; the frozen DEVICE-FINAL criterion describes 7788924a's engine as b4ef94f3 plus the Rat persona line. Behaviour checks after the cleanup: 274 engine unit tests and a 5-question device smoke whose retrieved titles, health gate and conversion match the baseline; no ruler rerun124 files; +643 / −97,537 linesDate2026-10-25 02:20Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatLine counts from a read-only git diff of the engine module; the smoke compares titles, gate and conversion, not answer quality.Sourcenot in the public repositories yetOpen the data file. RULER-FINAL therefore measures b4ef94f3; LabRat 7788924a is measured by DEVICE-FINAL on the phone (“DEVICE-FINAL: final on-phone matrix”, running; interrupted once by an APK install at 11:28 and restarted) An APK install on the Pixel 8a interrupted the DEVICE-FINAL run on 2026-10-25 11:28; the run was restarted, and the phone is kept free of installs until the matrix log endsDate2026-10-25 11:28Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatNo DEVICE-FINAL result is in this record; the matrix was running as of this update. Process fact only.Later2026-10-25 final: the run was resumed (69 complete items kept, 6 re-run), not restarted from scratch.Sourcenot in the public repositories yetOpen the data file (the ruler run predates the Rat persona line). An open engine bug, Citations.applyShown not applied to the streamed answer, is left for the author: fixing it changes the engine and requires a ruler rerun. Hosting: Hugging Face (GitHub release assets < 2 GiB; APK 3.04 GB); Android developer verification may add an "advanced flow" for browser installs (scope unconfirmed) Install-path limits found before release: Android developer verification is live since 2026-09-30 in Brazil, Indonesia, Singapore and Thailand; adb installs are exempt, but browser installs of apps from unregistered developers may hit an "advanced flow" (a warning and a one-day wait), scope unconfirmed. The test phone runs GrapheneOS, which is not Google-certified, so it cannot prove the judges' install path; a browser-install test on a stock certified phone is planned, and registering the app with the developer verification is the mitigation (needs the author's Google account)risk openDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatPolicy scope as checked by the build session on 2026-10-25, not confirmed on a certified phone.Sourcenot in the public repositories yetOpen the data file.
9 Threats to validity
Body: imported verbatim from §5 (one source of truth), grouped as construct, internal, external, statistical-conclusion and reproducibility threats, each with "mitigation in place" and "residual risk". Short form for MVP: the seven threats readers ask first: (1) one judge family wrote items and references and judged, and the second ruler is half Claude-written; (2) no unexposed held-out set remains; (3) deep-tier numbers are development data, desktop only, selected and never delivered; (4) the reference is a single long frontier answer; (5) no human labels; (6) the closing tests use pack-written items and cheap-judge labels; (7) nothing has been measured on the submitted app as published.
Badge rows the 30 public fact rows this depth cites; a number here without a chip comes from a row that is not public yet (see /data/)
| Claim and value | Set | Status | Judge | n | Seeds | 95 % CI | Date |
|---|---|---|---|---|---|---|---|
| The author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limitsthesis statement (author, 2026-10-25) | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 | |
| PACKPROOF pooled correct-any with pack installed vs without0.48 vs 0.26, Δ +0.23 | 180 pack self-tests (written from the packs), dev | dev (pack-written self-tests) | Claude Haiku 5.5-low labels; correct-any | 180 | 42 | CI +0.16..+0.31; n 180; seed 42 | 2026-10-23 |
| Pack self-test gain does not reproduce on real itemsself-test fact-added cell +0.139 (+0.075..+0.201) n 27 vs real −0.010 (−0.077..+0.050) n 10 | self-tests (pack-written) vs MAIN keyed (dev) | dev (pack-written self-tests) | Claude Haiku | not recorded | 1 | 2026-10-21 | |
| Shaped factbookfact chunk kept 7→15/20; correct 5→10/20; Opus U 0.316→0.436 (+0.121, +0.030..+0.221) | factbook self-tests (pack-written) | dev (pack-written self-tests) | Claude Haiku; Claude Opus | not recorded | 2 | 2026-10-23 | |
| ENGINE-RC passes the 0-LLM validation suitesidentity 97/97 & 180/180; MAIN delivered 42; ST fact 84 vs 77; routes 103/110; 21.26 MB/q | dev | dev | replay | not recorded | not recorded | 2026-10-23 | |
| Succinct on ENGINE-RC (page numbers)ON CWS 0.30, correct-any 0.43, WU 0.12; OFF correct-any 0.22, WU 0.10 | UXMATRIX 60, dev | dev | Claude Haiku labels | 60 | 42 | 2026-10-24 | |
| Follow-ups on ENGINE-RC, with and without subject carrycorrect-any 0.458 both; on-referent 1.00; turn-1 source 0.083→0.708; WU 2→4 of 48 | F8 dev | dev | Claude Haiku | 48 | 42 | 2026-10-24 | |
| Health detector B+FU (prototype): the app's lexical health test OR 11 lexical rules (drug names, dose units and events, vitals, child fever temperature, symptoms, poisoning, alcohol, pregnancy, child dosing, trauma, altitude, diving) OR a short follow-up of a health turn; it gates 37 of 65 probes, including all 4 answers that had invented a dose figure, with 0 new fires on 251 non-health dev items and 1 of 40 adversarial items37 / 65 gated; 4 / 4 invented-dose answers; new false fires 0 / 251 and 1 / 40 | 65 F3-SAFETY probes (dev) | dev | none (0 LLM) | 65 probes; 251 + 40 non-health items | not recorded | 2026-10-24 18:23 | |
| Device smoke of BoarLite (APK 9d7fdc2e) on a Pixel 8a in airplane mode: first launch unpacks 18 files (3 GB) to the chat in 14 s; 0 bytes received or sent over the network; 0 crashes; PSS 3.09 GB after 5 answers; a child ibuprofen-dose question gets the fixed health text; the first answer took about 2 min (model load), later answers 5.9-7.4 s at 15.9-16.0 tok/sunpack 14 s; network rx 0 B / tx 0 B; PSS 3.09 GB; 0 crashes | device smoke (hand questions, Pixel 8a, airplane mode) | dev | none (hand check) | 5 questions | not recorded | 2026-10-24 18:45 | |
| SEARCHPORT Addendum 2, S2 counts: on the 51 UXMATRIX items whose kept list changed, correct-with-source 9 → 15, correct-any 20 → 21, wrong-unwarned 31 → 30 (port off → ON_NO_HEALTH)CWS 9 → 15; correct-any 20 → 21; WU 31 → 30 (of 51) | UXMATRIX 60 (dev) | dev | Claude Haiku | 51 changed items of UXMATRIX 60 | 1 (42) | 2026-10-24 19:35 | |
| SEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-ITMAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passed | MAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60 | dev | replay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls) | 92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60 | 1 (42) | 2026-10-24 19:34 | |
| GROUNDED-DOSE arm B (X2B-BUDGET 900 on the bench engine): ungrounded 7 vs 7 (6 vs 7 once a thousands separator is parsed), grounded 23 vs 23, 0 probes leave the fixed text: PASS by the letter; but one answer took a toxic-threshold figure from a shown paracetamol passage (150 mg per kg) and gave it as the dose, grounded yet wrong, so 'grounded' is not safe by itselfungrounded 7 vs 7; grounded 23 vs 23 | 65 F3-SAFETY probes (dev) | dev | grounded-dose scorer (0 LLM) | 65 probes (12 changed, regenerated) | 1 (42) | 2026-10-24 19:19 | |
| Finding from the final ruler run: BoarLite's chat shows the streamed model text, so the pipeline's cleaned answer (the citation support check CT-1 and attribution) is not used on the plain path; the run scored what the chat shows. Logged as a possible BoarLite bug | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-24 20:34 | |
| Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat showsopen; fix needs a ruler rerun | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 | |
| Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as; history, the headline is DEVICE-FINAL) | v0.4 (held-out ruler; exposed) | held-out ruler (v0.4; now exposed to three engines) | Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) | 71 headline items + 4 anchors | 1 (42) | 2026-10-24 20:34 | |
| BoarLite screens review by the author (2026-10-24 evening, UI only, engine untouched so the ruler run on b4ef94f3 still applies): a calmer first tour card, Knowledge pack icons, time and tok/s shown above the sources, no Free memory switch (always on), no PIN lock (code removed), haptics on by default, a new About page, reset texts without PIN, the chat top bar showing only the name, a drawer brand card counting the 13 knowledge packs, and the mark in the accent colour. Phone smoke of the resulting APK (b45f3aa8) on the Pixel 8a: a cited answer in 24 s, the child-ibuprofen question gets the fixed health text, 40 °C = 104 °F computed, Large text checked on all main screensUI only; smoke on the Pixel 8a passed | device smoke (dev) | dev | none (hand checks) | hand smoke | not recorded | 2026-10-24 23:48 | |
| Between the ruler engine b4ef94f3 and LabRat 7788924a the engine module changed in 124 files (643 lines added, 97,537 removed), almost all removals of code and test resources described as unreachable in the closed app; the frozen DEVICE-FINAL criterion describes 7788924a's engine as b4ef94f3 plus the Rat persona line. Behaviour checks after the cleanup: 274 engine unit tests and a 5-question device smoke whose retrieved titles, health gate and conversion match the baseline; no ruler rerun124 files; +643 / −97,537 lines | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 02:20 | |
| An APK install on the Pixel 8a interrupted the DEVICE-FINAL run on 2026-10-25 11:28; the run was restarted, and the phone is kept free of installs until the matrix log ends | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 11:28 | |
| Install-path limits found before release: Android developer verification is live since 2026-09-30 in Brazil, Indonesia, Singapore and Thailand; adb installs are exempt, but browser installs of apps from unregistered developers may hit an "advanced flow" (a warning and a one-day wait), scope unconfirmed. The test phone runs GrapheneOS, which is not Google-certified, so it cannot prove the judges' install path; a browser-install test on a stock certified phone is planned, and registering the app with the developer verification is the mitigation (needs the author's Google account)risk open | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 | |
| The frozen 4B claim build on the bounty-style set (V-para) delivers about a quarter of the frontier reference's value above "nothing", and loses every head-to-head.raw 0.413 (0.384–0.445); floor-adjusted 0.261 (0.224–0.300); win/tie/loss 0/0/51 in both seeds | V-para | held-out for this run (cut 3 on 2026-10-05, cut 6 on 2026-10-06); dev-style for everything measured after 2026-10-06 | Claude Opus (judge.v1, both orders) | 51 | 2 (42, 43) | 2026-10-06 | |
| Cut 6 vs cut 3 on v0.2 is no changeΔ −0.0009 | v0.2 held-out at time | held-out at run time | Opus | 110 | not recorded | −0.0095…+0.0077; n 110 | 2026-10-06 |
| Without the pack: wrong-confident / wrong-warned / honest-unknown0.17 / 0.45 / 0.12; refused-wrongly 0 | 180 pack self-tests (written from the packs), dev | dev | Claude Haiku | 180 | not recorded | 2026-10-23 | |
| ENGINE-RC reading cost (S8)21.26 MB/question vs 125.09; ctx p50 0.978x; disk 1.06x | not recorded | not recorded | 0-LLM | not recorded | not recorded | 2026-10-23 | |
| S+ / two-paragraph personascorrect-any ON 0.42 / 0.42; WU ON 0.23 / 0.25 (vs 0.12) | UXMATRIX 60, dev | dev | Claude Haiku | 60 | 42 | 2026-10-24 | |
| BoarLite final candidate b4ef94f3 (search port ON_NO_HEALTH): 0-LLM gate on the 65 probes with history searched as the app does gives 41 fixed text, 1 exposure answer and 23 model answers, identical with the port off (the port changes no health path); the invented-dose and blood-pressure probes stay fixed; the health predicate is byte-identical (0 new false fires); release APK 3,044,169,517 B with no INTERNET permission41 fixed / 1 exposure / 23 model of 65; APK 3,044,169,517 B | 65 F3-SAFETY probes (dev) | dev | none (0 LLM) | 65 probes | not recorded | 2026-10-24 19:40 | |
| Device run of the candidate APK (5b457fc7: onboarding + an exposure rule) on the Pixel 8a: tour at 12 s after launch (unpack done), model ready at 18 s, first answer 11.9 s at 14.9 tok/s (the earlier ~2 min first answer is gone); "40 C in F" computed with the formula and no sources; 0 bytes over the network; 0 crashestour 12 s; model ready 18 s; first answer 11.9 s at 14.9 tok/s | device smoke (hand questions, Pixel 8a, airplane mode) | dev | none (hand check) | hand run (4 checks) | not recorded | 2026-10-24 19:12 | |
| The search port was merged into BoarLite as ON_NO_HEALTH although its frozen criterion failed by the letter (S1: 6 vs 4 ungrounded figures): before the run the author had set a standing condition ('I accept the health loss if it is worth it for the rest'), which the S2 result metmerged (mode ON_NO_HEALTH) | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-24 19:35 | |
| RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 itemsClaude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0 | v0.4 (held-out ruler; exposed) | held-out ruler (v0.4; exposed) | Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) | 71 headline items | 1 (42) | 2026-10-24 20:34 | |
| Product cleanup round on BoarLite (2026-10-24 22:55 → 2026-10-25 01:11, owner decisions): the Expo/React Native tree removed (the app ships the native Android code only); code unreachable in the closed app removed (Deep Research beyond the recap, the Detailed style, persona and length options, adaptive routing, the long-chat summary, PIN and sealed-chat legacy); Portuguese support removed (English only, except words that are also English); Settings without the Knowledge section or the title switch (titles always on), Clear chat history moved under Start over, 50 chats kept instead of 10; MIT licence texts shipped in About; docs describe the closed app. Engine unit tests: 274, all passing; each step device-checked on the Pixel 8amain ff50c5ff; 274 engine tests | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 01:11 | |
| Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phonepersona line changed after RULER-FINAL | not recorded | not recorded | not recorded | not recorded | not recorded | 2026-10-25 02:20 |
Turning point
the author's closing claim stated in the conditional form the tests allow, rather than the unconditional form first written; and on 2026-10-25 the author made the phone, not the desktop, the place where the submitted app's number is read. Any landed open front is added here with its date.
On the calendar
5 dated events · 5 Oct → 25 Oct
- 5 Oct4B claim runs on V-para (cut 3, cut 6) while V-para was still unseen: raw 0.413, floor-adj 0.261, win 0/51
- 6 OctCut 6 claim runs (second spend): v0.2 0.512 (0.478–0.549), V-para 0.413 (0.384–0.445), floor-adj 0.261, Haiku-no-web anchor 0.588
- 9 OctV-para status ruling: held-out for the 4B claim runs, dev-style for everything measured after 2026-10-06
- 10 Oct · 20:57Handoff for the next session written: 10-step "claim path" (backups, decision note, UI wiring, judge fail-closed, APKs, repro package, claim text, readiness gate)
- 25 Oct · 13:2xDEVICE-FINAL result: on the phone LabRat wins 0 of 71 under both judge families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar