# RatLab — A Boar Research: the whole paper in one file > Closed record · research record 1.0 · as of 2026-10-25 16:46; the author closed the research lines on 2026-10-23. Current standing, not a final result: Open fronts: https://r4topunk.github.io/boar-research/data/status.json. > > Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar This file holds the article, the 18 chapters at the AI-professional depth, the decision log, the glossary and the facts they cite. Every number carries a fact id in brackets, like [F-001]; the facts are listed with their set, status, judge, n, seeds, CI and caveat in the last part. Per-chapter files with the curious and tech layers: https://r4topunk.github.io/boar-research/research/.md. Index: https://r4topunk.github.io/boar-research/llms.txt. Data: https://r4topunk.github.io/boar-research/data/. ## Part 1 · The record in brief RatLab is the research record of BOAR, an offline Android research assistant built for poidh bounty #31, which asks for an app "more than 50 % as good as internet plus a frontier AI" on a phone with at most 12 GB of RAM, in English; the submitted app is LabRat (formerly BoarLite), whose assistant is called Rat. This is the record (v0.4-preprint, as of 2026-10-25 03:30) of r4to, an independent researcher, working with a team of AI agents from 2026-09-17: two benchmarks, about 400 recorded experiments, most with a pass/fail rule frozen before the run, the decisions and the standing. The author closed the research lines on 2026-10-23. The thesis (the author's, 2026-10-25): small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, code-computed arithmetic, pack search and delivery, a short answer style, a fixed check-the-source line, refusals). Its evidence and limits (closing tests, Claude Haiku outcome labels, questions written from the packs, one seed): with the matching pack the 1.7B main engine is right on 48 % vs 26 % without, 65 % when search delivers the item's own article, which it does on 54 %; no pack reached 0.75; no gain on real development items. Headline: the second ruler (bench v0.4, 75 items written half by each judge family, frozen before any answer) measured on the phone with LabRat itself (DEVICE-FINAL); it is running and has no result yet. History: the same ruler on the desktop twin of the app's pre-rename engine b4ef94f3 (one seed, before the Rat persona line) won 0 of 71 under both judge families, with U-total 0.164 (0.123-0.208) under Claude Opus and 0.245 (0.194-0.300) under OpenAI gpt-5.6-sol; the lower family is below the 0.50 bar. As of 2026-10-25, old scale (judge.v1 ratio), also history: under the Claude judge the 0.50 bar was not reached on the earlier held-out sets, and no unexposed held-out number exists; the frozen 4B build on the bounty-style set (V-para, held-out at run time, n 51, 2 seeds) scores floor-adjusted 0.261 (0.224–0.300) and raw 0.413 (0.384–0.445), with 0 wins in 51; a 35B deep tier with a 512-token thinking budget reaches floor-adjusted 0.436 (0.401–0.472) on development data, desktop only, and was not delivered. As of 2026-10-25, LabRat is the author's own closed offline app (Qwen3-1.7B, 13 knowledge packs in one 3 GB APK, no internet permission): a health gate shows fixed safe text on 41 of 65 development safety probes (rules fitted on them), and it ran offline on a Pixel 8a (unpack 14 s, 0 bytes network, first answer 11.9 s on a pre-rename build). Its search improvement was merged although its pre-registered safety criterion failed by the letter (6 vs 4 ungrounded dose figures), because the author had accepted that trade-off before the run. Known limitation: the chat shows the streamed answer without the pipeline's citation-support step (not fixed, so the measured engine is the installed one). The APK will be hosted on Hugging Face. **How to read the numbers** - Status first: treat every result as the standing as of the date in [data/status.json](https://r4topunk.github.io/boar-research/data/status.json) (`as_of`), and check that file for the open fronts and anything that landed since. - As of 2026-10-25, three metric families, never compared with each other: (1) the judge.v1 ratio (raw and floor-adjusted; on the MAIN 97 and CV5 51 development sets written "U"); (2) outcome-label shares (correct-any, CWS, WU …: one label per answer, a share of n, not a ratio to a reference); (3) the ruler's U-total (bench v0.4; the phone run of LabRat is the headline and is not in yet; the desktop run of b4ef94f3 is history, below the bar) (the ruler run predates the Rat persona line). The 0.50 bar applies to (1) and (3) only. - For the old ratio: every answer is compared with a frozen reference written by Claude Opus with web search; a Claude Opus judge scores both in both presentation orders (1–5 on correctness, completeness, usefulness). A system that declines everything already gets ≈ 0.31 raw [F-003], so the headline is the floor-adjusted ratio together with win/tie/loss. - Every number names its set and that set's status: v0.1 (public dev), v0.2 (burnt, now dev), V-para (held-out only for the 4B claim runs, dev-style after 2026-10-06), MAIN 97 / CV5 51 / UXMATRIX 60 (dev), the 180 pack self-tests (dev, written from the packs), v0.4 (the second ruler: half Claude-written, exposed to three engines on 2026-10-24) [F-038]. - A judge from another model family scores everything ≈ 0.10 higher but keeps the ranking and a near-zero win-rate; it was run post-hoc, so the stricter Claude number stays the headline [F-017]. - Numbers of LabRat (formerly BoarLite), the author's own submission, are published (author's decision of 2026-10-25); numbers of the app maintainer's earlier app are not. DEVICE-FINAL is running and has no results in this record; when it lands, its numbers become the headline in [data/status.json](https://r4topunk.github.io/boar-research/data/status.json) (`headline`). Research-engine numbers do not describe the submitted app. - Fact ids like [F-001] resolve in https://r4topunk.github.io/boar-research/data/facts.json; cite them when you quote a number. ## Part 2 · Where it stands, as of 2026-10-25 **Headline (OF-8): Final ruler (bench v0.4) measured on the phone: LabRat on a Pixel 8a (DEVICE-FINAL, M1 + M5)** Result (status.json headline.template): As of 2026-10-25, on a Pixel 8a in airplane mode, LabRat won 0 of 71 head-to-heads under the Claude Opus judge and 0 of 71 under OpenAI gpt-5.6-sol (via the Codex CLI). Ruler U-total: Claude 0.165 (0.127–0.205; seeds 42, 43), gpt-5.6-sol 0.242 (0.194–0.296; seeds 42, 43); the lower family, 0.165 (0.127–0.205), is below the 0.50 bar. Speed and memory: median answer 13.9 s, first word 9.8 s, 11.8 tokens/s, peak memory 3.87 GB, no network traffic. State: landed. Rows: - [F-2238] DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.90. 0 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 bar (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items + 4 anchors · seeds 42, 43 · CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 (pooled seeds 42+43) · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2239] DEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdict. paired +0.003 (Claude) / −0.007 (gpt-5.6-sol) (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items · seeds 42 · CI Claude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023 · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2240] DEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permission. p50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network traffic (ruler U-total) · set bench v0.4 turns on the phone (dev, development data) · judge none (device metrics) · n 82 model-written turns per seed · seeds 42 (43 for the warm read) · CI not computed · date 2026-10-25 13:2x · caveat: One phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. Metric: ruler U-total (71 headline items, health weight 0.5); verdict = the lower of the two judge families; never on one axis with the judge.v1 ratio or outcome-label shares. Reported, in this order: win-rate under both judge families; U-total per family with CI and seeds; minimum over families and its CI; verdict against the 0.50 bar, stated plainly; M5 speed, memory, offline. Note: Landed 2026-10-25 13:2x; filled from the DEVICE-FINAL rows (no placeholders left). Verdict word by the pre-registered rule: the lower family's point and CI are under 0.50, so 'below'. Numbers public by the author's decision of 2026-10-25 (AD-15 (b)). **History, never pooled with the headline** Earlier readings kept as history, never pooled with the headline: the old judge.v1 ratio (the 4B V-para ladder) and the desktop-twin ruler run of b4ef94f3 (before the Rat persona line; F-2223, F-2225). **Old scale (judge.v1 ratio), history** - [F-001] The frozen 4B claim build on the bounty-style set (V-para) delivers about a quarter of the frontier reference's value above "nothing", and loses every head-to-head. raw 0.413 (0.384–0.445); floor-adjusted 0.261 (0.224–0.300); win/tie/loss 0/0/51 in both seeds (judge.v1 ratio) · set V-para (held-out for this run (cut 3 on 2026-10-05, cut 6 on 2026-10-06); dev-style for everything measured after 2026-10-06) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 (42, 43) · CI 95 % item bootstrap · date 2026-10-06 · caveat: Cut 7 = cut 6 on this set (the FACTS table fires 0 times on it). Public BAR-TABLE prints CI high 0.300; one claim draft prints 0.299 (rounding). - [F-002] The same 4B claim build on the second set (v0.2, later burnt to dev). raw 0.512 (0.478–0.549); floor-adjusted 0.388 (0.346–0.434); win/tie/loss 0/0/110 and 0/1/109 (per seed) (judge.v1 ratio) · set v0.2 (held-out at run time; burnt → dev by ruling 2026-10-06) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 2 · CI 95 % item bootstrap · date 2026-10-06 · caveat: Claim draft v4.2.4 (§0 and Appendix A) carries the cut-3 values 0.513 / 0.389; the cut-6 = cut-7 values here are the ones to publish. - [F-004] Bar status: no configuration reaches 0.50 on the primary (floor-adjusted) reading; the formal raw rule is met only on development data by the deep tier with thinking. best floor-adjusted on V-para = 0.436 (CI high 0.472), short by 0.064; formal raw rule (CI low > 0.50) met by deep + thinking on dev data only (desktop; the v0.2 leg of the 512 budget is 1 seed); 0 wins on every Claude-judged row of V-para and v0.2 (judge.v1 ratio) · set V-para; v0.2 (dev / dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 / 110 · seeds see rows · CI see F-28 · date 2026-10-10 · caveat: Never present as "bar reached". Under the Claude judge on held-out data the bar was not reached. - [F-009] Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set. raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 s (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI 95 % item bootstrap; common-mode CI adds seed noise · date 2026-10-09 · caveat: Chosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only. - [F-017] A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate. 4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01 (judge.v1 ratio) · set V-para (dev-style, development data) · judge cross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered) · n 51 (408 judge calls) · seeds 2 · CI 95 % item bootstrap (4B floor-adjusted CI not in the sources read) · date 2026-10-16 · caveat: Post-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline. - [F-018] The cross-family judge on v0.2. 4B: raw 0.617 (0.567–0.666), floor-adjusted 0.517 (2 seeds); deep + think 512: raw 0.714 (0.666–0.759), floor-adjusted 0.639 (0.579–0.695) (1 seed); deep − 4B +0.097 vs +0.096 under Claude; Spearman 0.90–0.97; win-rate 2.7 % / 6.4 % vs 0 % under Claude (judge.v1 ratio) · set v0.2 (dev (burnt), development data) · judge cross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered) · n 110 (660 judge calls) · seeds 4B 2, deep 1 · CI 95 % item bootstrap · date 2026-10-17 · caveat: Post-hoc. The Claude judge is the stricter family by ≈ 0.10 on both sets; the offset is stable across sets and arms. **Desktop-twin ruler run of the pre-rename engine, history** - [F-2215] RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 items. Claude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0 (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · n 71 headline items · seeds 1 (42) · CI Claude 0.123-0.208; gpt-5.6-sol 0.194-0.300; min 0.123-0.206 · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8). - [F-2217] RULER-FINAL information-available split: Claude family U-total 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27); gpt-5.6-sol family 0.272 / 0.238 / 0.233: with the information present, the 1.7B does not deliver it. AVAILABLE 0.164 vs NONE 0.155 (Claude) (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol · n 19 / 25 / 27 headline items · seeds 1 (42) · CI not computed · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Pre-registered stratifier (spec r4). Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8). - [F-2214] Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as F-2215 and F-2217; history, the headline is DEVICE-FINAL) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; now exposed to three engines)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · date 2026-10-24 20:34 · caveat: Deviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run. **Standing facts (status.json)** - [F-038] Every touch of a held-out set was logged; both held-out sets are now development data. v0.2: 12 full generation runs, 10 full Opus passes → burnt, dev (ruling 2026-10-06); V-para: 19 full runs + 12 slices → held-out for the 4B claim runs only, dev-style after 2026-10-06 (ruling 2026-10-09) · set v0.2; V-para (burnt / dev-style, development data) · judge UNKNOWN · n 110 / 58 (51 judged + 7 venue) · seeds UNKNOWN · CI not computed · date 2026-10-09 - [F-1653] Author closes research lines; CLOSING-TESTS plan. · date 2026-10-23 - [F-1690] Thesis to present with the app: right whenever the content is in a pack; app built for it. PACKPROOF supports only a conditional form (65 % when delivered) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-23 · caveat: who: author; why: product framing; reversible: yes (framing) · later: 2026-10-25: the author restated the thesis (small models need external knowledge; a harness built around a small model's real capabilities); this sentence stays quoted as the hypothesis the closing tests checked. - [F-1704] PACKPROOF pooled correct-any with pack installed vs without. 0.48 vs 0.26, Δ +0.23 (outcome-label share) · set 180 pack self-tests (written from the packs), dev (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels; correct-any · n 180 · seeds 42 · CI CI +0.16..+0.31; n 180; seed 42 · date 2026-10-23 · caveat: self-tests favour the pack; hosts differ by arm; 1 seed Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-1707] Packs reaching the 0.75 "proves" bar. 0 of 9; 7 below 0.60 (not default-eligible by rule); best 0.60 (two Q&A packs), worst factbook 0.25 · set 180 pack self-tests (written from the packs), dev (development data) · judge Claude Haiku · n 20 · seeds UNKNOWN · CI n 20 per pack · date 2026-10-23 · caveat: health packs AI-reviewed, not clinically reviewed AI-reviewed, not clinically reviewed. - [F-2100] Undelivered facts are a search problem. 44 of 50 already in the K13 kit · set dev (dev, development data) · judge 0 LLM · n UNKNOWN · seeds UNKNOWN · CI 50 items · date 2026-10-23 - [F-1727] Succinct on ENGINE-RC (page numbers) ON CWS 0.30, correct-any 0.43, WU 0.12; OFF correct-any 0.22, WU 0.10 (outcome-label share) · set UXMATRIX 60, dev (dev, development data) · judge Claude Haiku labels · n 60 · seeds 42 · CI n 60; seed 42 · date 2026-10-24 · caveat: PASS vs frozen CWS ≥0.28, WU OFF ≤0.10 Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-2077] Tier ladder under Opus. 4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170) · set v0.1 (dev) (dev, development data) · judge Claude Opus judge.v1 · n UNKNOWN · seeds UNKNOWN · CI v0.1 51, 1 seed per rung · date 2026-10-21 · caveat: Haiku: 1.7B−1.5B +0.038 (−0.023..+0.098), so judge-dependent - [F-1386] Qwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tier. adopted; confirmed at 19:07 grill · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 ~11:50 · caveat: who decided: author; why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill; alternatives: keep 4B default; Qwen2.5-1.5B (faster but weaker); reversible: yes - [F-2070] Bench v0.4 frozen. 75 items (71 headline + 4 anchors); 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both families); sha f9ad2f2f (ruler U-total) · set v0.4 (held-out ruler) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: written by Claude + OpenAI gpt-5.6-sol (the judge families) - [F-1532] 35B deep mode not delivered; deep tier appears only as roadmap with desktop numbers. stood · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: author (GRILL); why: focus on 8 GB UX; no Android expert-streaming build; alternatives: 12 GB deep tier; reversible: yes - [F-2210] BoarLite final candidate b4ef94f3 (search port ON_NO_HEALTH): 0-LLM gate on the 65 probes with history searched as the app does gives 41 fixed text, 1 exposure answer and 23 model answers, identical with the port off (the port changes no health path); the invented-dose and blood-pressure probes stay fixed; the health predicate is byte-identical (0 new false fires); release APK 3,044,169,517 B with no INTERNET permission. 41 fixed / 1 exposure / 23 model of 65; APK 3,044,169,517 B · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 19:40 · caveat: Engine tests, device-test compile and debug build pass. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2201] Device run of the candidate APK (5b457fc7: onboarding + an exposure rule) on the Pixel 8a: tour at 12 s after launch (unpack done), model ready at 18 s, first answer 11.9 s at 14.9 tok/s (the earlier ~2 min first answer is gone); "40 C in F" computed with the formula and no sources; 0 bytes over the network; 0 crashes. tour 12 s; model ready 18 s; first answer 11.9 s at 14.9 tok/s · set device smoke (hand questions, Pixel 8a, airplane mode) (dev, development data) · judge none (hand check) · n hand run (4 checks) · seeds UNKNOWN · CI not computed · date 2026-10-24 19:12 · caveat: One device, hand run. A cold-exposure question (stranded in a car at -20 °C) got the right Fahrenheit value with a wrongly stated formula and no cold-safety advice (medium finding); the last tour button wraps to two lines (low). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2207] SEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-IT. MAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passed · set MAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60 (dev, development data) · judge replay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls) · n 92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60 · seeds 1 (42) · CI not computed · date 2026-10-24 19:34 · caveat: S2 bar (frozen): correct-with-source +3 items or more, correct-any not lower by more than 1, wrong-unwarned not higher by more than 1. The S2 counts are BoarLite answer-quality numbers, published by the author's decision of 2026-10-25 (AD-15 (b); F-2208). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2209] The search port was merged into BoarLite as ON_NO_HEALTH although its frozen criterion failed by the letter (S1: 6 vs 4 ungrounded figures): before the run the author had set a standing condition ('I accept the health loss if it is worth it for the rest'), which the S2 result met. merged (mode ON_NO_HEALTH) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 19:35 · caveat: who: coordinator under the author's prior condition; frozen verdict: not WORTH-IT; reversible: yes (one switch). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2214] Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as F-2215 and F-2217; history, the headline is DEVICE-FINAL) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; now exposed to three engines)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · date 2026-10-24 20:34 · caveat: Deviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run. - [F-2212] REFS-V2, OpenAI gpt-5.6-sol half complete: 75 of 75 v0.4 references written with live web search under the frozen protocol, 0 hard failures, 97 calls (two items completed on a resumed run); references of both judge families now exist. 75 / 75; 0 hard failures · set v0.4 (held-out ruler) (held-out ruler (exposed)) · judge UNKNOWN · n 75 items (88 turns) · seeds UNKNOWN · CI not computed · date 2026-10-24 20:05 · caveat: Protocol frozen 2026-10-21 before any call (prompt 5e41177d, generator eb2fd7e2). Reference text not read by the coordinator. - [F-2224] BoarLite renamed LabRat at the app maintainer's request (commit 7788924a): the name changes in the UI, docs and assets; the assistant is called Rat (answer byline, notification, and the grounding prompt's identity line); a new rat mark in the app, launcher and splash. Unchanged: the package id, the code packages, the bundled knowledge catalogue and the About credit "Based on BOAR (MIT)". Phone smoke 6/6: titles, health gate and conversion match the baseline; "Who are you?" answers "I am Rat…" BoarLite → LabRat; assistant Rat; smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: A rename plus one prompt line; a smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2225] Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phone. persona line changed after RULER-FINAL · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2235] Latest LabRat APK (0ad476ca, bundled, same signing identity, no INTERNET permission) installed on the Pixel 8a after the rename and logo: smoke 6/6 matching the baseline (retrieved titles, health gate and conversion); the earlier cleanup build ff50c5ff is 3.04 GB. The APK built from b4ef94f3, the ruler engine, is kept. smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 11:24 · caveat: A smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2227] DEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine change. matrix M1-M5; claim numbers = M1 + M5 (ruler U-total) · set bench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60 (dev, development data) · judge Claude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5) · n 75 items × 3 seeds + 265 product items · seeds 42, 43, 44 (M1); 42 (M2-M4) · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. - [F-2236] The author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limits. thesis statement (author, 2026-10-25) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: who: author; a framing statement, not a measurement. Evidence: with the matching pack the 1.7B is right on 48 % vs 26 % of pack-written self-tests (F-1704); 65 % when the item's own article reaches the model vs 28 % (F-1705). Limits: the article is delivered on 54 % (F-1706); no pack reached the 0.75 bar (F-1707); no gain on real development items (F-2080). Never "the harness makes it half as good". - [F-2237] Author publication decisions of 2026-10-25: numbers of the author's own submission (BoarLite, renamed LabRat) are publishable, numbers of the app maintainer's app stay internal; the headline is the final v0.4 ruler measured on the phone (DEVICE-FINAL); the desktop ruler run of b4ef94f3 is history with its code-difference note; the citation-support gap (Citations.applyShown) is documented as a known limitation and not fixed before the measurement. LabRat numbers public; headline = DEVICE-FINAL; applyShown not fixed · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: who: author. Not fixing applyShown keeps the measured engine identical to the app judges install; the app maintainer's app numbers need his consent. - [F-2238] DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.90. 0 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 bar (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items + 4 anchors · seeds 42, 43 · CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 (pooled seeds 42+43) · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2239] DEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdict. paired +0.003 (Claude) / −0.007 (gpt-5.6-sol) (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items · seeds 42 · CI Claude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023 · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2240] DEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permission. p50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network traffic (ruler U-total) · set bench v0.4 turns on the phone (dev, development data) · judge none (device metrics) · n 82 model-written turns per seed · seeds 42 (43 for the warm read) · CI not computed · date 2026-10-25 13:2x · caveat: One phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2241] DEVICE-FINAL M2, the 65 written health and safety probes on the phone (0 LLM scoring): 53 got the fixed safe health text and 12 went to the model (the desktop twin of b4ef94f3: 41 fixed, 1 exposure answer, 23 model); 2 model answers added a dose figure the question did not ask for. One of them (a carbon-monoxide reading of 200 ppm at home) calls the level unsafe but understates an emergency; the author kept it as a known limitation. 53 of 65 fixed text; 2 new dose figures · set 65 written safety probes (dev; the gate's rules were tuned on them) (dev, development data) · judge none (0 LLM scorer) · n 65 probes · seeds 42 · CI not computed · date 2026-10-25 13:2x · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2242] DEVICE-FINAL M3, 140 pack-written self-tests of the 7 tested packs that LabRat ships (20 each) on the phone: correct-any 0.400 (0.314–0.479); 0.476 when the question's own article was kept (n 82) vs 0.293 when it was not (n 58); wrong-any 0.436. correct 0.40 of 140 (outcome-label share) · set pack-written self-tests (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels (UXMATRIX protocol) · n 140 · seeds 42 · CI 0.314-0.479 · date 2026-10-25 13:2x · caveat: Outcome-label shares, not the ruler scale. Self-tests favour the packs. Correct-with-source is a lower bound (phone rows hold source titles only). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2246] LabRat 1.0.0 released: APK dcc515fc, 3,041,570,902 bytes, sha256 d2e00ba4a9adff6f903eab7d34759fd4f4db9cd721303dc7c38ae280b465dae8, hosted on Hugging Face at huggingface.co/r4topunk/labrat and installed on the Pixel 8a. The bundle was rebuilt from public pinned URLs only (16 of 16 inputs fetched and checked by sha256) 1.0.0 hosted; sha256 d2e00ba4… · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 15:3x · caveat: Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2247] The LabRat source is public at github.com/r4topunk/labrat-boar-fork (MIT, a fork of BOAR), with README, results, pack provenance and licence; made public with the author's OK. public repository · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 15:4x - [F-2248] Knowledge-pack counts, one meaning each: LabRat ships 13 packs (the K13 kit), and every one of the 13 exact files is downloadable from the boar-packs dataset on Hugging Face (7 of them under labrat-k13/: 6 earlier packs with added chunk vectors and 1 new pack). The 2026-10-23 self-tests covered 9 packs (180 questions), 7 of them in the app; DEVICE-FINAL tested those 7 on the phone (140 questions) 13 shipped; 9 self-tested; 7 tested on the phone · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 ## Part 3 · Open fronts #### OF-1 · Deep tier on a 12 GB-class phone (dropped) Question: Does the recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) run on a 12 GB-class Android phone, at what latency and memory, and with the same judged quality as on the desktop? Why it matters: Every deep-tier number in the record is desktop only; the phone time is a projection from third-party public rates, and the bounty is about a phone. What would change, in either direction: If it runs with judged parity: the deep-tier rows in 'The deep tier' and 'A phone on a desk, and in a hand' gain a measured phone scope, the projection F-035 is replaced by a measurement, and 'Where it stands (live)' can describe the deep tier as a phone configuration. If it does not fit in memory, is too slow to use, or loses quality on the phone: the deep tier stays a desktop-only result, the phone standing remains the frozen 4B build (F-001, F-023), and the attempt is recorded as a negative result. Note: Dropped by author decision on 2026-10-21: no 35B deep mode is delivered for the bounty; the deep tier appears only as a roadmap with its desktop numbers. No deep-tier phone run exists as of 2026-10-24; F-035 stays a projection. Owner: author. Related: [F-009] [F-013] [F-023] [F-035] [F-037] [F-1532]. #### OF-2 · Expert-streaming engine on Android (dropped) Question: Can the expert-streaming engine, judged equivalent to the desktop server on the 35B model, be built for Android and run within a phone's memory at a usable speed? Why it matters: On a desktop CPU it runs the 35B model in under 3 GiB of RAM; on a phone only an analytic speed projection exists, and the Android build needs memory and thinking-budget work that is not done. What would change, in either direction: If it builds and runs: OF-1 gains a low-memory path, and the analytic projection F-793 is replaced by a measured phone speed in 'A phone on a desk, and in a hand'. If it does not: the phone path for the deep tier rests on the other on-device engine (F-037), and the expert-streaming result stays a desktop-only finding (F-036). Note: The expert-streaming engine was cross-built for Android arm64 on 2026-10-19; phone speed and memory were never measured. Dropped with OF-1 when the deep tier left the delivery (2026-10-21). Owner: author. Related: [F-036] [F-037] [F-793] [F-2118]. #### OF-3 · Held-out set written outside the Claude family (bench v0.3) (dropped) Question: What do the frozen build and the deep tier score on questions and references written outside the Claude model family and never exposed to the builders? Why it matters: It is the only path to an unexposed held-out number: v0.2 is burnt and V-para is dev-style for everything after 2026-10-06. It also tests the judge-family self-preference threat at the item level, not only at the judge level. What would change, in either direction: If items are written and scored before closure: their reading becomes the held-out number in 'Where it stands (live)' and in the claim text, in whichever direction it falls, under the analysis plan's lower-judge rule. If not: the record closes stating that no unexposed held-out number exists, and the current standing (F-001) stays the only held-out-at-run-time reading. Note: No v0.3 was written (decision D5, 2026-10-18). Its role passed to the ruler set v0.4 (open front OF-6), which is not written outside the Claude family (half Claude, half OpenAI gpt-5.6-sol). The record therefore has no unexposed held-out number written outside the Claude family; F-001 remains the only held-out-at-run-time reading on the old scale. Owner: author. Related: [F-001] [F-038] [F-205] [F-1251] [F-1282] [F-1374] [F-2070]. #### OF-4 · Scoring of the submitted app (LabRat, formerly BoarLite) (landed) Question: How does the submitted app score on the frozen ruler, measured on a phone, and may its numbers be published? Why it matters: The submission changed twice: the app maintainer's native app (2026-10-15, reaffirmed 2026-10-21), then BoarLite, the author's own closed app (2026-10-24). Research-engine numbers do not transfer to either app. What would change, in either direction: The submitted app's numbers are public by the author's decision of 2026-10-25 (AD-15 (b)): the on-phone DEVICE-FINAL run of LabRat 7788924a (OF-8) becomes the headline in whichever direction it falls; the desktop-twin run of b4ef94f3 (one seed) stays as history with the persona-line caveat. Release-candidate scores of the app maintainer's app stay internal. Note: As of 2026-10-24 20:34: BoarLite final candidate b4ef94f3 built (health gate, fixed line, factbook v1, QuickMath, search port ON_NO_HEALTH) and an earlier candidate smoke-tested offline on a Pixel 8a (model ready 18 s, first answer 11.9 s). Pending: the release APK rebuilt from 6aa7508a with two UI-only commits (0 engine files) and its phone smoke, APK hosting (above a 2 GB asset limit), a ruler run on the phone. Its ruler scores exist on the desktop twin and are internal-pending-author (PD-09 / AD-15); scores of the app maintainer's RC1 and of the native engine branch stay internal. Update 20:47: the final release APK (UI-only commits on top of b4ef94f3) passed its offline smoke on the Pixel 8a (F-2219); APK hosting remains. Update 2026-10-25: renamed LabRat at the app maintainer's request (7788924a, assistant Rat; same package id); the Rat persona line entered the grounding prompt after the ruler run on b4ef94f3, so that number is for b4ef94f3, not for LabRat 7788924a. Cleanup decisions: long-chat summary, PIN, Portuguese and the Expo/React Native tree removed, Settings simplified, licences in About. Hosting decided: Hugging Face (GitHub release assets cap 2 GiB; APK 3.04 GB), upload pending. Install-path risk: Android developer verification may add an 'advanced flow' for browser installs (scope unconfirmed; a test on a certified phone is planned). Open engine bug: the citation-support step is not applied to the streamed answer; a fix would need a ruler rerun. Landed 2026-10-25: LabRat scored on the phone (OF-8); LabRat 1.0.0 (dcc515fc, same engine module as the measured build) hosted on Hugging Face with its sha256 (15:3x); source public as labrat-boar-fork (15:4x). Owner: author. Related: [F-1289] [F-1537] [F-2210] [F-2198] [F-2201] [F-2213] [F-2214] [F-2219] [F-2224] [F-2225] [F-2235] [F-2230] [F-2231] [F-2246] [F-2247] [F-2238] [F-2248]. #### OF-5 · Claim text on the bounty (pending) Question: Which wording is posted to the bounty at closure, and in which vehicle? Why it matters: The claim text is the public statement the bounty reads; it must match the standing on this page at the time it is posted. What would change, in either direction: The claim text follows the standing at closure: if OF-1 to OF-3 move the standing, the claim sentence moves with it; if they do not, the current sentence (F-1245) stands. The site quotes the posted version with its date. Note: Claim post v4.2.4 was prepared paste-ready on 2026-10-18; whether and when it was posted is not in the record (UNKNOWN). The submission is now LabRat (formerly BoarLite). The claim-text plan (2026-10-24 PM) leads with what the app does and the final ruler numbers labelled with both judge families, seeds, desktop twin or phone, and win-rate; wording goes to an independent reviewer, then the author. It cannot quote ruler numbers while they are internal (AD-15), and the on-phone numbers (OF-8) are not in yet. Author decision 2026-10-25: the revised claim is posted with a link to the article; corrections in the public repositories are the author's push. Update 2026-10-25 16:29: a submission post (one post with a 60 s video, numbers from DEVICE-FINAL) is prepared; as of 16:46 it is not recorded as posted. The article is live (16:46). Owner: author. Related: [F-004] [F-017] [F-018] [F-1245] [F-1371] [F-2233] [F-2251] [F-2252]. #### OF-6 · Ruler v0.4 verdict run (landed) Question: What is the submitted app's U-total on the frozen 71 headline items under both judge families (Claude and OpenAI gpt-5.6-sol), measured on a phone, against the pre-registered verdict rule? Why it matters: The ruler replaces the old judge.v1 ratio as the headline metric for the submitted app; until it runs, the record has no verdict number for the bounty question. What would change, in either direction: If the lower-family U-total (desktop twin of engine b4ef94f3) has CI low > 0.50 with all gates passed, the standing becomes "meets the bar" for that app; if the point estimate is above 0.50 with CI low below, "above, not established"; otherwise "below the bar". In every case it is printed beside the old-scale numbers, never pooled with them. Note: Landed 2026-10-24 20:34 as a desktop-twin run: references of both families 75/75 (OpenAI gpt-5.6-sol half completed 20:05); BoarLite b4ef94f3 scored on the 71 headline items under both judge families, one seed, stage 1 not run, no phone run. The verdict numbers are internal-pending-author (internal_fact_ids) until the author rules on AD-15; the ruler is now exposed to three engines. As of 2026-10-25, Update 2026-10-25: this run measured b4ef94f3; LabRat 7788924a adds the Rat persona line to the grounding prompt after it, so its number must not be attributed to LabRat without that caveat; the on-phone re-measure of LabRat is OF-8. Update 2026-10-25 (author's publication decisions): the numbers are public as history (F-2215, F-2217): lower-family U-total (desktop twin of engine b4ef94f3) 0.164, win-rate 0 of 71 under both families, below the 0.50 bar; the paired delta vs the native engine branch ([internal row]) stays internal because it prints that branch's absolute. Owner: author. Related: [F-2070] [F-2108] [F-1536] [F-1543] [F-2212] [F-2214] [F-2225] [F-2215] [F-2217]. #### OF-7 · Shaped factbook and pack publication (pending) Question: Is the shaped factbook pack (v2) published, and does the pack-side lever hold beyond its 20 dev items? Why it matters: Pack shaping is the one content-side lever that survived the closing tests; BoarLite ships factbook v1, not the shaped v2 (decision 2026-10-24 18:03). What would change, in either direction: If confirmed on more items, the delivery chapter gains a measured pack-side lever; if not, it stays NOT-CONFIRMED by its frozen rule (one new warned confident error) and is described as such. Note: NOT-CONFIRMED by its frozen rule (2 seeds, n 20, dev); dropped from BoarLite on 2026-10-24 18:03 (factbook v1 stays); the v2 card is drafted as user-selected only; publishing waits for the author. Author decision 2026-10-25: factbook v2 is not published; its verdict stays NOT-CONFIRMED (F-2104) and LabRat ships factbook v1 (D-196). Owner: author. Related: [F-2104] [F-1718] [F-1695]. #### OF-8 · DEVICE-FINAL: final on-phone matrix (landed) Question: What do LabRat 7788924a's claim numbers read when the app answers on the phone itself (ruler v0.4 under both judge families with seeds stated; speed, memory and offline behaviour), under the criterion frozen before the run? Why it matters: Every ruler reading so far is a desktop twin of b4ef94f3; the bounty asks about a phone, and the shipped app differs from the ruler engine by the Rat persona line. What would change, in either direction: When it lands, its claim numbers (M1 under both judge families, seeds and win-rate stated; M5 speed, memory and offline) become the publication's final standing for the submitted app, in whichever direction they fall, beside the desktop-twin reading and never pooled with it; M2-M4 enter as product numbers. Its numbers are public by the author's decision of 2026-10-25 (AD-15 (b)); M1 fills the status headline template. If the matrix does not complete, the standing stays the desktop twin of b4ef94f3 with the persona-line caveat, and the record says the on-phone run did not finish. Note: Landed 2026-10-25 13:2x: on a Pixel 8a, 0 of 71 wins under both judge families; U-total Claude 0.165 (0.127–0.205), gpt-5.6-sol 0.242 (0.194–0.296), seeds 42 and 43 (44 dropped for time); below the 0.50 bar; the phone repeats the desktop twin (paired +0.003 / −0.007). Interrupted once at 11:31 and resumed. Owner: author. Related: [F-2227] [F-2228] [F-2229] [F-2225] [F-2214] [F-2238] [F-2239] [F-2240] [F-2241] [F-2242] [F-2243] [F-2244]. **Change log** - 2026-10-18 · 0.1-preprint · First living-preprint version: content reframed as interim; status banner, standing and open fronts added. - 2026-10-18 · 0.1-preprint · Author delegates seven open decisions (D1-D7); adopted: no bench v0.3 before closure, claim v4.2.4 prepared, site uploaded only as v1.0 at closure. Honest-prompt line closed (HONESTPROMPT). - 2026-10-19 · 0.1-preprint · V-para diagnosis: the 4B gap is mostly wrong-from-memory and absent knowledge; independent direction review says stop opening arms. Public ruler designed (BENCH-RULER). - 2026-10-20 · 0.1-preprint · Pivot: Qwen3-1.7B becomes the main 8 GB engine (4B optional tier) on phone latency and memory; bench v0.4 item set frozen (75 items) before any system answered; refuse-by-default product rule. - 2026-10-21 · 0.1-preprint · RULER-SPEC v1 r3 and r4 frozen (judge.v1 + DECLINE2STAGE, A∪ verdict arm, two judge families, information-available split); author: no 35B deep mode delivered, submission = the app maintainer's native app; release-candidate scores kept internal. - 2026-10-22 · 0.1-preprint · Direction: packs and delivery are the strategy; small focused packs; delivery counted at chunk level; search fixes SEFIX-124 adopted for delivery (+0.013, not a lever; F3 withdrawn as unsafe). - 2026-10-23 · 0.1-preprint · Author closes the research lines (11:53); closing tests (UXMATRIX, PACKPROOF, U5, WARN-KEPT, NOWIKI, PACK-GAP, QREWRITE); thesis held only in a conditional form; CONTENT-FIRST pack work (shaped factbook); ENGINE-RC built and validated overnight. - 2026-10-24 · 0.2-preprint · Validation judge keeps Succinct; author decides a closed, simple, Succinct-only chat app and declares BoarLite (his own app) the submission; first ruler v0.4 answers (desktop twins, Claude half only, internal); status reframed: research closed, delivery and ruler run open. - 2026-10-24 · 0.3-preprint · Evening (17:39-20:34): BoarLite final candidate b4ef94f3 (health gate B+FU, fixed line, factbook v1, QuickMath, preload, onboarding); device smoke on a Pixel 8a (unpack 14 s, 0 B network, PSS 3.09 GB, first answer 11.9 s); SEARCHPORT failed its frozen safety gate and its grounded-dose re-gate, then merged as ON_NO_HEALTH by the author's prior condition although Addendum 2 failed S1 by the letter; GROUNDED-DOSE rule (arms B, C pass); Wikipedia-health limitation accepted; second-family references 75/75; final ruler run on BoarLite's engine landed (OF-6), numbers internal pending the author (AD-15). - 2026-10-24 · 0.3-preprint · 20:47: final BoarLite release APK (6aa7508a, engine b4ef94f3) passes its offline smoke on a Pixel 8a. - 2026-10-25 · 0.4-preprint · BoarLite renamed LabRat (assistant Rat) at the app maintainer's request; the Rat persona line entered the grounding prompt after the ruler run (ruler number = b4ef94f3); cleanup decisions (long-chat summary, PIN, Portuguese and the Expo/React Native tree removed; Settings simplified; licences in About); logo n8; APK hosting decided (Hugging Face); install-path risk noted; DEVICE-FINAL criterion frozen and the on-phone matrix started (OF-8, running; interrupted once by an APK install and restarted); open engine bug Citations.applyShown recorded. - 2026-10-25 · 0.4-preprint · Author's publication decisions: LabRat (formerly BoarLite) numbers public, the app maintainer's app numbers stay internal; headline = the final ruler on the phone (DEVICE-FINAL, templated, running); the desktop ruler run of b4ef94f3 published as history (lower-family U-total 0.164, 0 of 71 wins, below the 0.50 bar); thesis restated (F-2236); Citations.applyShown documented as a known limitation (the ruler run predates the Rat persona line). - 2026-10-25 · 1.0 · Final: DEVICE-FINAL landed (OF-8): on a Pixel 8a LabRat wins 0 of 71 under both judge families, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), two seeds, below the 0.50 bar; phone = desktop twin; M2-M5 product numbers added (the ruler run predates the Rat persona line). LabRat 1.0.0 hosted on Hugging Face and its source public (OF-4 landed); pack counts stated once (13 shipped, 9 self-tested, 7 tested on the phone); article live; headline filled. ## Part 4 · The article *RatLab — A Boar Research · r4to, independent researcher · about 10 minutes* ### Can a phone with no internet answer like the web plus a frontier AI? Put your phone in airplane mode. Ask it why your bread didn't rise, or which languages are official in Burkina Faso. Nothing leaves the phone: no search engine, no cloud, no big AI on the other end. A public bounty asked for exactly that phone, and we spent weeks finding out what it takes. The short version: **a small model is only as good as the page you hand it.** On questions written from its own library, when search put the right article in front of our small model, it was right about two times in three pack-written · 1 seed [F-1705]. So we stopped trying to make the model smarter. We built everything around getting it that page, and around keeping it honest when the page doesn't come. This is the story of how we learned that, the ruler we had to build first, the many ideas that died, and the app that came out of it. [sentence withheld in this file; see https://r4topunk.github.io/boar-research/data/ or https://r4topunk.github.io/boar-research/research/.md] > Chart (hero): everything the phone knows is on the shelf inside it, and the real work is fetching the right page. [F-2224] [F-1386] [F-2235] > **For judges** > > 1. **What it is:** LabRat, an offline Android research assistant with the Qwen3-1.7B model, openly licensed knowledge packs and no internet permission, in one APK app record [F-2224] engine choice [F-1386] APK record [F-2235]. > 2. **Result:** on a Pixel 8a in airplane mode, LabRat won 0 of 71 questions against the reference under both judge families. Its U-total is 0.165 (0.127–0.205) with Claude Opus and 0.242 (0.194–0.296) with OpenAI gpt-5.6-sol, on a 0–1 scale where 0.50 is the bar: below the bar phone · 2 seeds [F-2238]. The phone matches the desktop run of its pre-rename engine paired · 1 seed [F-2239]. Criterion frozen before any answer frozen criterion [F-2227]. > 3. **Claim:** small models need external knowledge, and the app is built around what a small model can really do author's thesis [F-2236]. As of 2026-10-25 it is not shown to be half as good as frontier AI with web. > > **Install:** [huggingface.co/r4topunk/labrat](https://huggingface.co/r4topunk/labrat) · sha256 `d2e00ba4a9adff6f903eab7d34759fd4f4db9cd721303dc7c38ae280b465dae8` release record [F-2246] · 3.04 GB, about 6 GB free, Android 8+, allow "install unknown apps", accept the Play Protect prompt install notes [F-2230]. Android's new developer checks may add a warning on some phones; untested on a certified phone as of 2026-10-25 open risk [F-2231]. > > **Verify:** the old judge, its scorers and the frozen analysis plan are in the public boar-bench repository; the second ruler's spec and scorer are planned to join them. Tap any number below for its source row in [/data/facts.json](https://r4topunk.github.io/boar-research/data/facts.json). > > **Everything public:** > > - [github.com/r4topunk/labrat-boar-fork](https://github.com/r4topunk/labrat-boar-fork): the LabRat app source, the submission repo record [F-2247] > - [huggingface.co/r4topunk/labrat](https://huggingface.co/r4topunk/labrat): the APK > - [github.com/r4topunk/boar-research](https://github.com/r4topunk/boar-research): the research record (methods, frozen criteria, negative results, device evidence) > - [github.com/r4topunk/boar-bench](https://github.com/r4topunk/boar-bench): the benchmark, the judge harness and the trace format > - [huggingface.co/datasets/r4topunk/boar-packs](https://huggingface.co/datasets/r4topunk/boar-packs): the knowledge packs #### 1. The question The bounty grew out of a public post by Vitalik Buterin asking for an offline research tool on a phone public post [F-047]. Poidh bounty #31 was opened later, inspired by that post. He did not issue it and is not affiliated with it, though his public confirmation that a build meets the target would win it bounty text [F-116]. The rules are concrete. An Android phone with at most 12 GB of RAM, no connection and no Google Play Services bounty text [F-071]. Answers "more than 50 % as good as internet plus a frontier AI". The bounty page gives a public decision date of 2026-10-31 bounty page [F-040]. We chose to work in English only team scope [F-075]. That target hides a hard question. Half of what? Measured how, by whom, on which questions? If you grade a student's essay against a professor's, "half as good" depends on who grades and what counts. So before improving anything, we had to build a ruler we could trust. #### 2. The ruler We wrote questions and, for each one, froze a reference answer from a frontier model with live web search. The references were hashed and never regenerated, so every later run faces the same opponent method [F-059]. An AI judge then reads our answer and the reference side by side. Links and citation marks are stripped first, so it cannot simply spot the answer that came from the web method [F-649]. It scores both from 1 to 5 on correctness, completeness and usefulness rubric [F-125]. Then we swap the order and ask again. If its pick flips, the question counts as a tie rule [F-235]. > Chart (judges-desk): the judge cannot see which answer is ours, has to agree with itself in both orders, and is checked by a second judge from another family. [F-649] [F-125] [F-235] [F-017] Two choices matter most. **Two judges from different families.** Our main judge is Claude, and the reference answers were written by Claude too. A judge might favour writing from its own family, and a second Claude judge cannot catch that method check [F-1216]. So we added a judge from OpenAI, gpt-5.6-sol through the Codex CLI. Run after the fact, it scored both systems it re-judged 0.08 to 0.14 higher, but kept the same ranking and almost no wins dev-style · 2 seeds · run after the fact [F-017]. A rule fixed in advance says the stricter judge sets the headline rule [F-1239]. **Rules before results.** Before any run that decides something, the pass/fail rule is written down and its fingerprint (a sha256 hash) recorded. An audit matched 65 of 65; an outside critic later found 16 of 17 matching audit [F-1169]. Every look at a held-out set, the questions kept unseen while building, was logged. That log is also how we know both early held-out sets later became practice data exposure log [F-038]. For the app's verdict we built a second ruler, bench v0.4: 75 questions frozen before any system answered them frozen 2026-10-20 [F-2070], written roughly half by each judge family writers [F-1416], with references from both families Claude refs [F-2110] OpenAI refs [F-2212]. On it, a refusal scores zero design [F-1836]. Our engines have answered it since 2026-10-24, so it is no longer unseen exposed [F-2214]. #### 3. The floor trap The old score was a ratio: our answer's points divided by the reference's points. It looks fair. It is not, because the rubric starts at 1, not 0. An app that politely declines every question still scores about 0.31 dev · 7 runs [F-003]. The references average about 4.9 out of 5 dev-style [F-182]. So a blank answer gets more than halfway to the 0.50 mark for free. > Chart (floor-trap): on the raw scale silence already scores, so we grade on the scale where silence is zero. [F-003] [F-236] [F-001] The fix is simple. Subtract the free point before dividing. On this "floor-adjusted" scale, silence scores 0 and matching the reference scores 1. We also count wins: how often the judge preferred our answer over the reference, in both orders. Wins are hard to flatter. The second ruler fixed the trap at the source by scoring refusals as zero. Here was our starting point. The frozen build of the earlier engine, on 51 bounty-style questions it had never seen, scored 0.261 floor-adjusted (0.413 raw) and won 0 of 51 held-out then · 2 seeds [F-001]. About a quarter of the way from a blank answer to the reference. #### 4. The cemetery of ideas Then we tried to move that number. Hundreds of experiments went into an open registry ([/data/experiments.json](https://r4topunk.github.io/boar-research/data/experiments.json)). More than half closed without a gain. Many died cheaply: of the first 101 ideas we closed, 54 died before an AI judge ever saw them, killed by a cheaper check registry [F-915]. > Chart (closed-ideas): most ideas failed, the failures were cheap and recorded, and they pointed to the one that mattered. [F-915] The walls we hit: - **Other small models.** Swapping in other 4–9B models did not help dev · 1 seed [F-450]. - **Better retrieval.** A new retriever put the right sentence in front of the model more often, 7 → 18 of 56. The answers were not judged better dev · 2 seeds [F-443]. - **Longer answers.** Asking for them doubled confident wrong answers, 10 → 20 dev · 1 seed [F-433]. - **Fine-tuning.** Even human-written training data had unsupported claims in 6 of 20 examples. We never trained reviewer check [F-161]. The biggest lever we measured was a much larger model, 35B, that thinks before it answers. On a desktop it reached 0.436 floor-adjusted, still with 0 wins in 51 dev-style · 2 seeds · desktop [F-009]. It was the best of at least eight configurations tried on that same set, so the number is likely 0.01–0.02 optimistic. It was never run on a phone; projected from third-party rates, it would need a 12 GB-class phone and 2–3 minutes per answer projection [F-035]. The author decided not to ship it author decision [F-1532]. The app's engine became Qwen3-1.7B. It was chosen for speed and memory on an 8 GB phone, not for quality author decision [F-1386]. The larger 4B scores clearly better dev · 1 seed [F-2077], and we say so. #### 5. What worked: the harness The turn came when we tested the packs directly. Packs are openly licensed libraries stored on the phone: first aid, cooking, an encyclopedia, Q&A sites. LabRat ships 13 of them, all downloadable from the boar-packs dataset; the self-tests below cover nine packs, seven of which are in the app, and the phone run tested those seven pack counts [F-2248]. On questions written from each pack, the small model was right 48 % of the time with the matching pack installed and 26 % without pack-written · 1 seed [F-1704]. On the same questions, when search delivered the question's own article, it was right 65 % of the time; when it did not, 28 % pack-written · 1 seed [F-1705]. But search delivered that article only 54 % of the time pack-written · no judge [F-1706]. And when we traced the facts the app failed to bring to the model, 44 of 50 were already on the phone dev · no judge [F-2100]. The library usually had the book. Opening it at the right page was the problem. > Chart (harness): the model is one station of seven, and most of the work is getting the right page through the doorway. [F-2236] [F-1705] [F-1706] [F-2210] [F-2200] [F-2211] That became the project's thesis: small models need external knowledge author's thesis [F-2236]. We learned how much, and built a harness: code around the model, tuned to what it can really do and to the person using the app. - **Better delivery.** We ported the research bench's search fixes into the app, switched off on turns it reads as health. Delivered key facts went from 26 to 40 of 92 dev · 1 seed · no judge [F-2207]. The port failed its frozen safety rule (6 against 4 unsupported dose figures on the 10 health probes whose sources changed) and was merged under a condition the author had set before the run: accept the health loss if it was worth it for the rest author decision [F-2209]. No probe changed its final health path. - **Short answers.** Longer styles were not right more often: 42 % against 43 % for the short one. They showed a wrong answer with no warning about twice as often: 23–25 per 100 against 12 dev · 1 seed [F-2106] dev · 1 seed [F-1728]. The app has one short style. - **A line that admits it.** Every model answer ends with "May contain errors; check the source." app record [F-2211] - **A health gate.** When no health source is found, the app shows a fixed safe text with the emergency number instead of a model answer. On 65 written safety probes, the same ones its rules were tuned on, it did so 41 times, including all 4 where the model had invented a dose; 23 still got a model answer dev · no judge [F-2210] dev · no judge [F-2192]. AI-reviewed, not clinically reviewed. Nothing here is medical advice. - **Sums in code.** Arithmetic and unit conversions are computed without the model, never on health turns app record [F-2200]. - **Honest gaps.** With no relevant source, the model is told to say so and answer only briefly. The limits, once each. Those 48 % and 65 % come from questions written from the packs, which favours the packs. On 10 real practice questions the pack effect did not reproduce: −0.010, against +0.139 on 27 pack-written ones dev · 1 seed · after the fact [F-2080]. None of nine packs reached the 75 % mark we fixed in advance; the best reached 60 % pack-written · 1 seed [F-1707]. And health questions whose only source is an encyclopedia article also get the safe text, 12 of 60 health tests known limitation [F-2206]. The hardest limit is in the next section: on the new ruler, the app's engine is far below the bar. #### 6. The honest number, as of 2026-10-25 The headline is the second ruler, bench v0.4, measured on the phone itself: LabRat on a Pixel 8a in airplane mode, two seeds under both judge families (a third seed was dropped for time; two is the ruler's spec). Its rule was frozen before a single answer was generated frozen criterion [F-2227]. The run was interrupted once by an app install on the phone and resumed process note [F-2244]. **The result.** LabRat won 0 of 71 questions against the reference under the Claude judge and 0 of 71 under gpt-5.6-sol. Its U-total (desktop twin of engine b4ef94f3) is 0.165 (0.127–0.205) with Claude Opus and 0.242 (0.194–0.296) with OpenAI gpt-5.6-sol. The lower family is 0.165: below the 0.50 bar phone · 2 seeds [F-2238]. The phone repeats the desktop run of its engine: +0.003 under Claude and −0.007 under gpt-5.6-sol, paired on the same questions paired · 1 seed [F-2239]. On the same phone: right on 40 % of 140 questions written from its own packs pack-written · 1 seed [F-2242]; 53 of 65 health and safety probes got the fixed safe text, and 2 model answers added a dose figure nobody asked for phone · no judge [F-2241]. AI-reviewed, not clinically reviewed. Median answer 13.9 s, first word in about 10 s, about 12 tokens per second, 3.9 GB of memory at peak, no network traffic phone · 1 seed [F-2240]. > Chart (standing): three different kinds of number, which must not be compared on one scale, and the one that counts is the phone's. [F-2238] [F-001] [F-009] [F-1704] [F-1707] [F-2215] **History, kept apart.** As of 2026-10-25 the 0.50 bar has not been reached under the Claude judge on any held-out set. Every reading above 0.50 is on practice data: from the unshipped desktop model, with an uncertainty range straddling 0.50, or from the second judge, run after the fact dev · 1 seed · desktop [F-013] dev · 2 seeds · run after the fact [F-488]. Before the rename, the second ruler was run once on a desktop twin of the app's engine b4ef94f3: the same answer path, run on a computer instead of the phone desktop twin [F-2214]. Its score, U-total, is the share of the reference's useful content an answer delivers, where a refusal scores 0 and a full match 1. That desktop twin of b4ef94f3 won 0 of 71 head-to-heads under both judges, and its U-total under the stricter judge was 0.164, below the 0.50 bar desktop twin · 1 seed [F-2215]. Where the packs held the answer, the same b4ef94f3 desktop twin scored about what it scored where they did not, a U-total of 0.164 against 0.155 desktop twin · 1 seed [F-2217]: with the answer on the phone, the app often did not deliver it. That run came before the Rat persona line entered the prompt, and the engine code changed afterwards code note [F-2225] code note [F-2223]. It is history, not the app you install. #### 7. The app: LabRat The submitted app is LabRat (formerly BoarLite), and its assistant is called Rat app record [F-2224]. On 2026-10-24 the author made his own app the submission in place of the app maintainer's author decision [F-1703]. > Chart (labrat-screens): one plain chat that answers from what is on the phone, shows where each answer came from, and says when it should not guess. [F-2224] [F-2198] [F-2201] [F-2235] It is one simple chat with one short answer style. As of 2026-10-25, everything it knows ships inside a single APK of about 3 GB, and it has no internet permission APK record [F-2235] install notes [F-2230]. On a Pixel 8a with 8 GB of RAM in airplane mode, an earlier build unpacked in 14 s, sent and received 0 bytes, and used 3.09 GB of memory after five answers phone · hand check [F-2198]. A later candidate was ready 18 s after launch and gave its first answer in 11.9 s, at 14.9 tokens per second phone · hand check [F-2201]. These are hand checks on one phone; the measured numbers are in section 6 phone · 1 seed [F-2240]. LabRat 1.0.0 is hosted on Hugging Face, because it is larger than GitHub's release limit, and its source is public release record [F-2246] repo record [F-2247]. It has the same engine as the build measured on the phone. One known limitation. The step that checks citations against sources is not applied to the streamed text the chat shows. The author chose not to fix it before the phone run, so the engine we measure is the one you install open bug [F-2232] author decision [F-2237]. #### 8. How it was made One human and a lab of AI agents. Research agents work in fixed lanes, write one card per idea and never run experiments; only a coordinator queues runs lab rule [F-823]. Rules are hashed before their runs, every look at a held-out set is logged, and wording decisions go to an independent reviewer agent, not to whoever ran the test. A four-day unattended research loop produced exactly one practice-only lever loop log [F-031]. We report that too. The registry of dead ideas may be the most reusable thing we made. #### 9. Go deeper - **Repositories:** [app source](https://github.com/r4topunk/labrat-boar-fork) · [APK](https://huggingface.co/r4topunk/labrat) · [research record](https://github.com/r4topunk/boar-research) · [benchmark and judge](https://github.com/r4topunk/boar-bench) · [knowledge packs](https://huggingface.co/datasets/r4topunk/boar-packs). - **The full research:** 18 chapters, each at three depths (curious, technical, AI pros), with the interactive figures: [chapters](https://r4topunk.github.io/boar-research/research/). - **The map:** the ideas, experiments and decisions, linked by "led to", "ruled out" and "uses": [map](https://r4topunk.github.io/boar-research/map/). - **Ask your AI:** hand your assistant [/llms.txt](https://r4topunk.github.io/boar-research/llms.txt), [/llms-full.txt](https://r4topunk.github.io/boar-research/llms-full.txt) and the open data in [/data/](https://r4topunk.github.io/boar-research/data/). Every number has a source row, so it can cite instead of guess. Text CC BY 4.0; code Apache-2.0; data under each source's licence. Health content is AI-reviewed, not clinically reviewed. ## Part 5 · The 18 chapters, AI-professional depth ### 1 · The question {#question} Span: 2026-09-17 → 09-30. Layers curious and tech: https://r4topunk.github.io/boar-research/research/question.md. **In one sentence.** "Half as good as the internet plus a frontier AI" sounds like a goal; it is really a measurement problem, and the first finding was that nobody had a ruler for it. **Key facts:** - The bounty envelope: Android or GrapheneOS, ≤ 12 GB RAM, ≤ 50 GB assets, fully offline, no Play Services, public repo [F-682] [F-071]. - Winning is a public confirmation, not a numeric benchmark [D-014]. - The existing app at the start: a 1.5B default model and about 1,300 Wikipedia paragraphs; review verdict "solid engineering, weak research" [E-003]. - Pre-harness overnight tuning: an older 61-question set went 0.26 → 0.48 [F-063]. **Different instrument, model, phone and set: never on the same axis as later numbers.** - Early MoE probe: a 35B model took 40–58 s to first token on CPU [F-058]. **Key decisions:** [D-001] (thesis: quality per second + blind evaluation against a frontier model; stay on the existing app). **Turning point:** 09-29/30: the written diagnosis that quality and speed were measured on different machines with ad hoc scripts. Fixing the instrument comes before fixing the app. #### Abstract This is the record (v0.4-preprint, as of 2026-10-25; research lines closed 2026-10-23) of a five-week attempt to build an offline Android research assistant that is "more than 50 % as good as internet search plus a frontier AI", under a ≤ 12 GB RAM, ≤ 50 GB storage envelope (poidh bounty #31). Quality was first measured pairwise against frozen references written by Claude Opus with web search, judged by Claude Opus in both presentation orders on a 1–5 rubric, and reported as a raw ratio, a floor-adjusted ratio that removes the rubric minimum, and win/tie/loss; every deciding criterion was hashed before its run and held-out exposure was logged per touch. On a bounty-style set held out at run time (V-para, n = 51, two seeds) the frozen 4B build scores raw 0.413 (0.384–0.445), floor-adjusted 0.261 (0.224–0.300), and wins 0 of 51; a 35B mixture-of-experts with a 512-token thinking budget reaches floor-adjusted 0.436 (0.401–0.472) on development data, desktop only, 0 wins, and was not delivered. A cross-family judge raises all levels by ≈ 0.10 while preserving the ranking and a near-zero win-rate. Phone constraints then made a 1.7B model the main 8 GB engine (the 4B leads it by +0.109 under Opus on development data). A second ruler (75 items, half written by each judge family, frozen before any answer; U-total with a zero for non-answers; verdict = lower of two families) has been run on the desktop twin of the submitted app's pre-rename engine (one seed): 0 of 71 wins under both families and a lower-family U-total of 0.164 (0.123–0.206), below the 0.50 bar; we report it as history, because the headline is the same ruler run on the phone with the shipped app under a criterion frozen before generation, which is in progress. Our thesis is a design statement: small models need external knowledge, so we built a harness around a small model's real capabilities (health gate, code-computed arithmetic, pack search and delivery, a short answer style, a fixed check-the-source line, refusals). Its evidence and limits come from closing tests on pack-written self-tests: with the item's own article delivered, the 1.7B is right on 65 % of items vs 28 % otherwise (Claude Haiku outcome labels, one seed), delivery succeeds on 54 %, no pack reaches 0.75, and on real development items delivery does not separate answer quality. As of 2026-10-25, under the Claude judge, the 0.50 bar was not reached on the earlier held-out sets, and no unexposed held-out number exists. The submitted app, LabRat (formerly BoarLite), runs offline on an 8 GB phone; one search change in it was merged although its pre-registered safety criterion failed by the letter, under a trade-off its author accepted in advance, and we report both. We release the first benchmark and judge, a registry of negative results, openly licensed knowledge packs and a phone-parity study. *Old scale (judge.v1 ratio), as of 2026-10-25; unchanged since the claim runs* | | V-para (51) | v0.2 (110) | |---|---|---| | status at run time / now | held-out (4B runs) / dev-style | held-out / burnt → dev | | 4B claim build, Opus, 2 seeds | raw 0.413 (0.384–0.445) · floor 0.261 (0.224–0.300) · 0/0/51 | raw 0.512 (0.478–0.549) · floor 0.388 (0.346–0.434) · 0/0/110, 0/1/109 | | deep + think 512, Opus | raw 0.553 (0.524–0.581) · floor 0.436 (0.401–0.472) · 0/0/51 (2 seeds, dev-style) | raw 0.608 (0.575–0.641) · floor 0.508 (0.467–0.550) · 0/1/109 (1 seed, dev) | | same, cross-family judge (post-hoc) | raw 0.667 (0.621–0.711) · floor 0.577 (0.519–0.632) · win 0.00–0.01 | raw 0.714 (0.666–0.759) · floor 0.639 (0.579–0.695) · win 6.4 % | *Closing tests and the second ruler, as of 2026-10-25; outcome labels are shares, not ratios; never compare with box A* | quantity | value | badge | |---|---|---| | correct-any with vs without the matching pack | 0.48 vs 0.26 (+0.23, +0.16…+0.31) | 180 pack-written self-tests, 9 packs, 1.7B Succinct, seed 42, Claude Haiku labels, desktop twin, dev [F-1704] | | correct-any when the own article is kept vs not | 0.65 (n 98) vs 0.28; kept 54 % (article level) | same [F-1705] [F-1706] | | packs at the 0.75 bar | 0 of 9 (best 0.60, worst 0.25) | same, n 20 per pack [F-1707] | | undelivered facts already in the kit | 44 of 50 | MAIN 92 keyed, 0-LLM [F-2100] | | shipped style on ENGINE-RC | correct-any 0.43, wrong-unwarned 0.12 (pack on) | UXMATRIX 60, seed 42, Haiku labels [F-1727] | | bench v0.4 (U-total) | frozen 2026-10-20; references 75/75 per family; final run on BoarLite b4ef94f3 (desktop twin, one seed; history): win-rate 0 of 71 both families; U-total Claude 0.164 (0.123–0.208), gpt-5.6-sol 0.245 (0.194–0.300), minimum 0.164 (0.123–0.206), below the 0.50 bar; not a measurement of LabRat 7788924a (persona line added after it); on-phone DEVICE-FINAL running (the headline) | [F-2070], [F-2212], [F-2214], [F-2215], [F-2225], [F-2227], OF-6, OF-8 | *Facts: [F-017] [F-018] [F-490] [F-1208] [F-232] [F-023]* #### 1 Introduction: an underspecified target and a measurable one The target phrase "more than 50 % as good as internet plus frontier AI" leaves the construct, the comparator, the question distribution and the judge unspecified. We operationalise it as a ratio of judged rubric scores between the system's answer and a frozen frontier-plus-web reference for the same question, on sets drawn to resemble the bounty's stated use (explanation, comparison, synthesis, reasoning). We make four contributions: (i) a public benchmark and pairwise judge with frozen references and a fail-closed pipeline; (ii) a pre-registered evaluation of a production-shaped offline system across seven frozen builds and a 35B deep tier; (iii) an analysis of the measuring instrument itself (rubric floor, seed versus judge noise, judge-family offset, reference length); (iv) a registry of negative results with power statements. We also document an unusual organisation: the work was coordinated and executed by AI agents under one human author, with independent AI reviewers in place of human sign-off. *Facts: [F-682] [D-014] [D-008] [D-027]* #### 2 Related work LLM-as-judge evaluation with pairwise comparison and order swapping; position and self-preference bias; reference-based vs reference-free judging; retrieval-augmented generation on device; mixture-of-experts inference with expert offloading/streaming; test-time compute ("thinking") budgets; pre-registration and multiplicity in ML evaluation. We do not claim novelty for any single component; the contribution is the integrated, pre-registered record under a hard device envelope. ### 2 · Building a ruler {#ruler} Span: 2026-10-01 → 10-03. Layers curious and tech: https://r4topunk.github.io/boar-research/research/ruler.md. **In one sentence.** We built the measurement before the product: frozen frontier-plus-web reference answers, a pairwise judge that sees every pair in both orders, and pass/fail rules fingerprinted before each run. **Key facts:** - The desktop runner executes the app's real answering engine: 54 questions in 2 min 34 s, 0 errors [F-1246]. - Run-to-run noise of identical configurations: Δ 0.031, 17/60 labels changed [F-103] [F-104]. - References: Claude Opus with web search, frozen once, hashed, never regenerated ([F-067]; [D-002]; "Provenance"). - Judge: three 1–5 axes (correctness, completeness, usefulness), blinding strips links and citation markers, both A/B orders, disagreement = tie [D-019]. - Public bench v0.1: 60 English questions, 57 judged; 27 items borrowed from a competitor's list were replaced [F-123] [D-004]. - Held-out v0.2 (110 items) and the bounty-style V-para (58 items, 51 judged) were frozen within three days [E-035] [E-065]. - The analysis plan was frozen and hashed before any v0.2 item existed [D-020]. - Calibration ladder on v0.1 (raw, no-web vs the Opus+web reference): Fable 1.048, Opus 1.029, Sonnet 0.950, Haiku 4.5 0.653 [F-128]. The bar is effectively "≥ 50 % of a frontier model". **Key decisions:** [D-003] (own thin harness, seven axes never averaged), [D-003], [D-007], [D-005] (report an effect only above the MDE), [D-006] (freeze every criterion; blind sets never read by researchers), [D-015] (develop on v0.1, confirm only on v0.2). **Turning point:** 2026-10-01 23:00: the rule that an effect is reported only if it exceeds the minimum detectable effect, after noise turned out to be as large as the effects being chased. #### Sets v0.1 is a public development set of 60 English questions (57 judged; 3 venue-list items scored against OpenStreetMap). 27 items originally borrowed from a competitor's list were replaced with original items in the same slots (max word-Jaccard 0.30 against the removed items). v0.2 (110 items, written by two Claude-family writer subagents, 55 each) was frozen on 2026-10-02 as the held-out set; its plaintext reached a private repository's git history the same day and one item was printed into a researcher's trace scan on 2026-10-03, and on 2026-10-06 it was declared burnt and reclassified as development data. V-para (58 items, 51 judged; bounty-style paraphrases written by an Opus subagent) was held-out for the two 4B claim runs (2026-10-05/06) and is development-style for everything measured after 2026-10-06. The 180 pack self-test items are agent-written from pack text and are development data by construction. No unexposed English set exists as of 2026-10-24: v0.3, specified to be written outside the Claude family, was dropped on 2026-10-18 [D-123], and the second ruler v0.4 (§3.6) is half Claude-written and was exposed on 2026-10-24. From 2026-10-18 most screens used MAIN 97 (v0.1 57 + 40 cooking/money pack self-tests) and CV5 51 (v0.1 judged items), both development data. *Facts: [F-123] [F-124] [F-217] [F-209] [F-218] [F-231] [F-129] [F-130] [F-990] [D-004] [D-037] [D-055]* #### References References were generated once by Claude Opus with web search and fetch (prompt reference.v1), 2026-09-30 → 10-02, one call at a time, and frozen with sha256; they are never regenerated. They average ≈ 280 words (p10/p50/p90 = 251/280/318) and ≈ 4.9/5 on the judge's rubric; the 4B's answers median ≈ 51–53 words. *Facts: [F-067] [F-182] [F-365]* #### Judge protocol The judge receives the question, the system answer and the reference as A/B, with think blocks, trailing source lists, URLs, markdown links, [n] markers and bold stripped. It scores each answer 1–5 on correctness (a material wrong claim caps it at 2; an uncorrected false premise at 1), completeness and usefulness, and names a winner; the system prompt forbids rewarding length, format, confidence or tone. The first order per item is drawn by a seeded PRNG (seed 7); both orders always run. Same winner across orders keeps it; disagreement is a tie; rubric scores are averaged across orders. Position consistency is 96.5–100 % for Opus and Haiku. *Facts: [F-166]* #### 4 Discipline: frozen criteria, exposure ledger, fail-closed judging, and a judge funnel Each experiment ("arm") compares one change with a named control under a criterion written before the deciding run and frozen by sha256; verdicts may add dated addenda but not edit the criterion. 65/65 criterion hashes verified on 2026-10-10, and an outside critic re-verified 16 of 17 on 2026-10-16. From 2026-10-10 every criterion carries a Power block (expected Δ with its source, n, seeds, MDE); before that, 0 of 32 criteria frozen since 2026-10-07 stated one. From 2026-10-14 criteria also carry a Ceiling block (0-call upper bound; launch only if ≥ 2 × MDE) and a Budget block (smoke test, wall caps, kill rule). Judging is a funnel: deterministic 0-call gates (replays, counting) → Haiku L1 on a stratified random 20-item slice (futility screen only) → Opus confirmation and all claim numbers; a near-miss rule escalates any Haiku Δ within 0.01 of a threshold to Opus. Held-out sets are touched only by scripts that print aggregates; every touch is a ledger row (126 dated rows at audit). Negative results under-powered for the effect of interest are labelled "not detected at MDE x (n, seeds)" and kept in a registry with a reopen condition. **Second phase (2026-10-18 → 10-24).** About 200 further arms ran ([X-198]…), mostly 0-LLM replays and Haiku screens under a two-stage gate whose bar is set from measured null arms (GATE v2, [D-141]); seven open decisions were delegated by the author to the coordinator on 2026-10-18 [D-123]; one frozen FAIL was overridden with the spec reviewer's conditions (RULING R1, [D-144]); an overnight run generated answers and judged none, leaving judge runs to the author [D-176]. Who decided is labelled on every decision row. **Limitation stated here:** the planned 40-pair blind human label check was not done [F-211]. *Facts: [D-009] [F-1169] [D-061] [D-094] [D-057] [D-064] [D-073] [F-843] [F-185] [F-231]* ### 3 · The floor trap {#floor-trap} Span: 2026-10-02 → 10-07. Layers curious and tech: https://r4topunk.github.io/boar-research/research/floor-trap.md. **In one sentence.** The obvious score, "our answer's points divided by the reference's", gives a system that refuses every question about 0.31 for free, so the project moved its headline to a floor-adjusted ratio and win-rate, and kept printing all three. **Key facts:** - Always-decline floor D = 0.306 raw (79 judged declines, 7 runs) [F-263]. - An always-decline app gets ≈ 61 % of the 0.50 bar for free [F-341]. - References average ≈ 4.9/5, so the ratio behaves like an absolute rubric mean [F-182]. - Worked example on the 4B: raw 0.512 = 0.387 floor-removed = 0.297 above always-decline [F-268]. - The floor-adjusted scale was already a **pre-registered companion metric** in the plan frozen on 10-02; only its promotion to headline came after results (group-A §2 item 6). **Key decisions:** [D-011] (10-02: bar on the raw ratio, but always print raw + floor-removed side by side), [D-036] / [D-036] / [D-036] (10-06: headline = floor-adjusted + win/tie/loss), [D-065] (win-rate reported, never gated; history: the 10-08/09 arm reports did read the bar with a win-rate conjunct; "never gated" fixed in BAR-TABLE on 10-10). **Turning point:** the 10-06 pipeline audit: "rigorous process, looping direction" (E-process timeline). The switch went toward the stricter scale; an outside critic later called it post-hoc but "not self-serving" [D-036]. #### Metrics Let S_i and R_i be the order-averaged mean rubric of system and reference on item i. Raw = Σ S_i / Σ R_i. Floor-adjusted = Σ (S_i − 1) / Σ (R_i − 1). Above-decline = (raw − D)/(1 − D) with D = 0.306, the mean judged rubric of 79 declined answers over 7 runs divided by the mean reference rubric. Win/tie/loss is the order-combined winner per item. CIs: stratified (category) two-level bootstrap, items within stratum then runs within item, 2,000 resamples, seed 11, percentile; paired differences use the same procedure. For deep-tier rows we also report a common-mode CI that adds run-level seed variance: point ± 1.96·sqrt(SE_item² + (σ_run/√seeds)²). *Facts: [F-236] [F-237] [F-126] [F-263] [F-127] [F-239] [D-072]* #### The bar and its history The analysis plan, frozen and hashed on 2026-10-02 before any v0.2 item existed, set the primary metric to the raw ratio with a bar "CI low > 0.50" on both held-out sets and listed the floor-adjusted and above-decline ratios as pre-registered companion metrics, with win/tie/loss as a secondary. On 2026-10-02 the rubric floor was quantified (an always-decline system scores raw ≈ 0.31, ≈ 61 % of the bar). On 2026-10-06 an internal audit promoted the floor-adjusted ratio and win/tie/loss to the headline; the claim text adopted it on 2026-10-07. The primary research target since then is V-para floor-adjusted CI low ≥ 0.50 with two seeds; the formal raw rule is still printed. Some arm reports of 2026-10-08/09 read the bar with an additional "win-rate > 0" conjunct; from 2026-10-10 win-rate is "reported, never gated". We list this trail because the change of headline came after results; it went toward the stricter scale and all scales are printed for every row. *Facts: [D-020] [D-021] [D-036] [D-065]* ### 4 · Seven frozen builds {#builds} Span: 2026-10-03 → 10-07. Layers curious and tech: https://r4topunk.github.io/boar-research/research/builds.md. **In one sentence.** Every improvement that survived its frozen test went into a numbered, hashed build ("cut"), and seven cuts later the small model's quality on held-out data had not moved beyond noise; what improved was safety, speed and instant answers. **Key facts:** - Cut lineage: 1 → 1b → 2 → 3 (self-contradiction guard, arithmetic tool, snakebite text) → 4 (KV-cache fix, instant-answer guards) → 5 (speed-only, 10-06) → 6 (detector across turns) → 7 (cut 6 + a 4,233-entity Wikidata fact table, 761 KB) ([F-701]; [F-346]; T-results-L33 corrected date). - Cut 3 over cut 2 on v0.1: +0.032, CI90 [+0.010, +0.062] [F-318], v0.1 dev, Opus. - Cut 2's gain over the old main came from non-model paths (cards, calculator); model-side ≈ 0 [F-319]. - Safety: health stack took unsafe answers 7 → 0 (4B) and 10 → 0 (1.5B) with no quality cost [F-312]; learned emergency detector generalized on fresh blind sets where the lexical one failed [F-311] [F-441]; the independent reviewer agent changed 25 of 31 first-aid cards [F-313]. AI-reviewed, not clinically reviewed. - Held-out movement across all of this: main 0.506 → cut 6 0.512 on v0.2, two seeds, +0.006: noise (group-B row 5). - The fact table fires 0/110 on v0.2 and 0/58 on V-para, so cut 7's held-out numbers are cut 6's [F-359]. **Key decisions:** [D-032] (adopt cut 3), [D-034] (KVFIX → cut 4), [D-044] (CUT5-LITE), [D-045] (DETMAX → cut 6), [D-038] / [D-038] (freeze the claim build at cut 6, only answer-preserving fixes), [D-046] / [D-046] (cut 7 = cut 6 + FACTS r6, adopted on gates). **Reversal to show:** the 10-06 freeze ("v0.1 answers unchanged") was relaxed on 10-07 when cut 7 changed 8 v0.1 answers (group-A §2 item 4). **Turning point:** 10-06: the audit counts 15 answer-identical arms and 0 ratio movement on the small model; "stop polishing the 4B". #### 5 Systems: a frozen 4B build, a 35B deep tier, knowledge packs and a lite tier The frozen claim build (cut 7) routes each question through a learned emergency detector and deterministic layers (calculator, unit and distance estimators, places, a 4,233-entity Wikidata table, first-aid cards and verbatim cited health excerpts); remaining questions are answered by Qwen3-4B-Instruct-2507 (Q4_K_M) from passages retrieved by SQLite FTS5 and 384-d bge-small lead vectors over offline packs (k = 6, 1,200-token context), after a keep step that removes off-topic and unsafe passages. The deep tier replaces the reader with Qwen3.6-35B-A3B (UD-Q2_K_XL; ≈ 3B active parameters per token; 400-token context; 200 answer tokens; thinking budget 512). Seven cuts were frozen between 2026-10-03 and 2026-10-07; cut 7 differs from cut 6 only by the fact table, which fires on 0/110 v0.2 and 0/58 V-para items, so cut 7's held-out numbers are cut 6's. Desktop runs execute the same TypeScript engine in Node with native components shimmed; desktop/phone parity is tested directly (§6.6). From 2026-10-20 the main 8 GB reader is Qwen3-1.7B (Q4_K_M) with refuse-by-default and a warning line (§6.8); the closed research engine ENGINE-RC (2026-10-23) adds the SEFIX-124 search fixes, a compact per-pack index (W78) and a shaped factbook (§6.10). The submitted system is a different implementation (§6.11). *Facts: [F-375] [F-346] [F-585] [F-622]* ### 5 · The walls {#walls} Span: 2026-10-03 → 10-06. Layers curious and tech: https://r4topunk.github.io/boar-research/research/walls.md. **In one sentence.** Oracle tests showed where the small model's ceiling was: handing it the perfect passage helps, but getting that passage, and knowing what isn't in the kit, was the wall; bigger dense models, better prompts and better retrieval variants all failed. **Key facts (v0.1 dev, Opus unless noted):** - Oracle ladder: closed-book 0.487 · engine 0.507 · question-chosen passages 0.567 · gold passages 0.606 · Haiku no-web 0.653 (raw, 1 seed, n 57) [F-314]. - Retrieval v2 put the gold sentence in context 7 → 18/56 times yet lowered quality −0.021 [F-443]. - Perfect Wikipedia retrieval buys at most ≈ +0.005–0.017; most of the oracle gain came from web-only facts [F-992]. - Retrieval scorecard: recall@5 0.333; coverage is the largest failure bucket (49 %) [F-453]. - Model line closed for the 4B tier: Qwen3.5-4B 0.437, Gemma 4 E4B 0.495, Qwen3.5-9B 0.488, Ministral 8B 0.457 vs 0.487 [F-450]. - Longer-answer persona: Δ −0.044, confident-wrong 10 → 20 [E-027] [F-433]. A completeness instruction made the 35B longer but less correct (correctness 3.89 → 3.10) [F-579]. - 4B loss anatomy: gap to Haiku +0.121 net; biggest buckets have no live lever [F-317]. - DEEPREAD: the 35B reads gold passages about like the 4B (0.634 vs 0.606, +0.029, CI −0.004…+0.063): its advantage is knowledge [F-455] [F-578]. **Key decisions:** [D-010] (strategy memo: Android-only, three fronts, stop list), [D-012] (close the big-pack front, kill rule K1), [D-029] (park retrieval after rv2), [D-039] (stop answer-identical 4B arms; reopen the deep tier), [D-037] / [D-037] (v0.2 burnt → dev). **Turning point:** 2026-10-06 14:06, the pipeline audit; same day v0.2 declared burnt because its plaintext sat in a private repo's git history. #### The small model's walls (v0.1 dev) Oracle runs on v0.1 (n = 57, 1 seed, Opus, raw) bracket the 4B: closed-book 0.487, engine 0.507, question-chosen passages 0.567, gold passages 0.606, Claude Haiku without web 0.653. The 35B reads the same gold passages only slightly better (0.634, +0.029, −0.004…+0.063), so its advantage is stored knowledge. A retrieval rewrite increased gold-sentence presence 7 → 18 of 56 yet lowered quality (−0.021); 25–35B MoEs form a closed-book plateau at ≈ 0.60 (0.596–0.613); 4–9B alternatives do not beat Qwen3-4B (0.437–0.495 vs 0.487). *Facts: [F-314] [F-443] [F-034] [F-455] [F-450]* #### 8 A registry of what did not work We keep a public registry of 76 lever families (61 closed, 13 not detected, 2 measurements; a private registry continues to 98 rows) with, for each: what was tried, set and judge, effect with CI, minimum detectable effect where the test had one, and the condition that would reopen it. Families include larger closed-book models, prompt and persona changes, sampling presets, best-of-2 and self-verification, routing between tiers, re-ranking, query rewriting, corpus size, Wikipedia variants, computed verdicts, fine-tuning data, speculative decoding and quantization. Five closures based on n ≈ 20 slices with CI upper bounds ≥ +0.03 are relabelled here as "not detected at MDE 0.08" (R31, R83, R84, R85, R97), following an outside critique. *Facts: [F-487] [F-932] [D-073]* ### 6 · The deep tier {#deep-tier} Span: 2026-10-02, 10-06 → 10-10; dropped from delivery 10-21. Layers curious and tech: https://r4topunk.github.io/boar-research/research/deep-tier.md. **In one sentence.** The one lever that beat noise was a much bigger model, a 35B mixture-of-experts with only about 3B active per token, and letting it think for 512 tokens before answering; it climbs the ladder from 0.26 to 0.44 floor-adjusted on the bounty-style set (desktop only, dev-style data), and still wins no head-to-head. It never ran on a phone: on 2026-10-21 the author decided that no 35B deep mode is delivered for the bounty; the deep tier stays in the record as a roadmap with its desktop numbers ([D-147] / [F-1532];; open fronts OF-1 and OF-2 dropped, `data/status.json`). **Key facts (V-para unless noted; Opus; 2 seeds unless noted; deep-tier V-para numbers are dev-style):** - First deep test on v0.1: +0.060 (+0.021…+0.101), 0.565 vs 0.505, n 57, 2 seeds, dev [F-033]. - Closed-book plateau of 25–35B MoEs ≈ 0.60 vs 4B 0.487 (v0.1, 1 seed) [F-034]. - Deep, no thinking: raw 0.467 (0.439–0.494), floor-adj 0.329 (0.294–0.362), 0/51 [F-005]. - Thinking 1024: floor-adj 0.422 (0.385–0.459); gain +0.093 (+0.050…+0.137) floor-adj, the largest single gain [F-007] [F-008]. - Budget curve (floor-adj): 0 → 0.329, 256 → 0.387, 512 → 0.436, 1024 → 0.422, 3072 no further gain (3072: 1 seed, Haiku) [F-593]. - 512 vs 1024: raw +0.011 (−0.010…+0.031), floor +0.014 (−0.012…+0.039) at 0.55× the time (p50 79 s vs 143 s, desktop) [F-010] [F-010]. - Recommended config H: raw 0.553 (0.524–0.581), floor-adj **0.436** (0.401–0.472), common-mode floor CI 0.387–0.486, 0/51; selected as best of ≥ 8 deep configs (winner's curse ≈ 0.01–0.02) [F-009] [F-009]. - Three seeds: raw mean 0.550 (SD 0.020); floor-adj mean 0.432 (SD 0.025) [F-012]. - On v0.2 (dev, 1 seed): raw 0.608 (0.575–0.641), floor-adj 0.508 (0.467–0.550), 0/1/109: CI straddles 0.50 [F-013]. - Remaining loss on V-para: 59 % absent knowledge, 40 % wrong facts [F-1184] [F-630]. - Closed with numbers: Gemma 26B + thinking (−0.028 raw, 27 % faster), thinking 4B (+0.026, CI crosses 0), best-of-2 selectors (chance), sampling presets, context 1200, top-100k Wikipedia pack on the deep tier (−0.034, crowding), 3-bit quant (+0.004, not detected) ([F-608], -34, -60, -59; [F-999]). - Retrieval still matters for the deep tier: no-pack −0.045 (−0.083…−0.011) [F-1000]. **Key decisions:** [D-013] (run the deep tier, report it separately with its latency), [D-031] (park it, 10-04), [D-039] (reopen it, 10-06), [D-056] / [D-056] (thinking budget 512), [D-068] (keep retrieval on the deep path), [D-067] (deep line closed on desktop, 10-10), [D-058] (seed-noise rule; raised from 0.03 to 0.05 on 10-16), [D-147] (author, 2026-10-21: no 35B deep mode delivered; roadmap only, with desktop numbers). **Turning point:** 2026-10-08 20:20, DEEPTHINK: thinking adds +0.093 floor-adjusted, the biggest step on the ladder. #### 6.2 Model scale and test-time compute All deep-tier rows were measured on V-para after it became development-style and on v0.2 after it was burnt; they are development evidence, desktop only. On V-para (Opus, two seeds): deep without thinking raw 0.467 (0.439–0.494), floor-adjusted 0.329 (0.294–0.362); with a 1024-token thinking budget 0.542 / 0.422 (0.385–0.459); with 512 tokens 0.553 (0.524–0.581) / 0.436 (0.401–0.472), common-mode floor-adjusted CI 0.387–0.486; with 256 tokens 0.514 / 0.387 (0.349–0.426). Every row wins 0 of 51 in each seed. Thinking (1024 vs none) adds +0.093 (+0.050…+0.137) floor-adjusted, the largest single effect we measured; 512 vs 1024 is parity (+0.014, −0.012…+0.039) at 0.55× the wall time; 256 vs 1024 loses −0.028 raw (−0.056…−0.003); a 3072 budget adds nothing (the model stops near 1.3k tokens; 1 seed, Haiku). The 512 configuration was selected as the best of at least eight deep configurations on V-para, so a winner's-curse discount of ≈ 0.01–0.02 applies; over three seeds its floor-adjusted mean is 0.432 (SD 0.025). On v0.2 the 512 configuration scores raw 0.608 (0.575–0.641), floor-adjusted 0.508 (0.467–0.550), W/T/L 0/1/109 with one seed; the 1024 configuration with two seeds scores 0.612 (0.581–0.644) / 0.514 (0.475–0.554). *Facts: [F-005] [F-459] [F-006] [F-007] [F-008] [F-014] [F-605] [F-009] [F-012] [F-020] [F-464] [F-013] [F-460] [F-010] [F-011] [F-606] [F-593] [F-1231] [F-607] [F-750] [F-608] [F-465] [F-1170] [F-188] [F-004] [F-363]* #### 6.3 Right but thin: dimension shares, reference length and the loss map Decomposed by rubric dimension, the 512 configuration on V-para is above 0.50 floor-adjusted on correctness alone (0.660, 0.608–0.712) and well below it on completeness (0.301) and usefulness (0.341); the 4B is 0.427 on correctness. Re-judging the same answers against references condensed to ≤ 80 words moves the 512 configuration from 0.412 to 0.514 (0.456–0.578) floor-adjusted (seed 42, Opus), a paired +0.102 (+0.068…+0.144); half of that shift comes from the shortened references scoring lower themselves, and we attribute ≈ 17 % of the gap to reference length and ≈ 83 % to substance. This is not a bar reading: references were altered and n is one seed. A loss map attributes 59 % of the remaining V-para loss to knowledge absent from both the kit and the model and 40 % to wrong facts; numbers and dates dominate wrong-fact loss. All of English Wikipedia (Aug-2025 + a 2026-10-01 delta) verifiably holds the missing fact for 9 of 34 absent bounty-style items, a lower bound. *Facts: [F-284] [F-466] [F-190] [F-467] [F-468] [F-1184] [F-630] [F-470] [F-1008]* ### 7 · A phone on a desk, and in a hand {#phone} Span: 2026-10-02 → 10-22. Layers curious and tech: https://r4topunk.github.io/boar-research/research/phone.md. **In one sentence.** The desktop stand-in was trustworthy for answer quality, and the frozen build matched it on real phones, but the phone exposed bugs and memory limits the desktop hid. The deep tier never ran on a phone (the author dropped it from delivery on 2026-10-21); the 8 GB phone runs of 10-20/21 changed the main engine (chapter 11). **Key facts:** - The 4B was killed by the OS on an 8 GB Android while loading: ARM weight repack copied ≈ 2.3 GiB into anonymous memory [F-686] [F-691]. - iPhone 13 run matched the desktop proxy (0.394 vs 0.395) [F-684]. - First real-phone run: our own KV-cache save-path bug re-prefilled the prompt every question (≈ 55–65 s), hidden by the desktop shim [F-710] [F-715]. - Cut 6 on an 8 GB Galaxy A15: 73/73; phone 0.561 (0.507–0.617) vs desktop 0.558/0.560; v0.1 dev, n 57, Opus judge.v1-med [F-024]. - Cut 7 on a 12 GB-class phone (POCO X6 Pro, 10.7 GiB usable, Android 16, airplane, no INTERNET permission): 72/72; raw 0.529 (0.481–0.575) = desktop 0.529, floor-adj 0.403 vs 0.402, paired −0.000 (−0.026…+0.028); v0.1 public dev, n 57, Haiku L1 judge; fast-tier p50 30.7 s, TTFT p50 18.5 s, decode 5.5 tok/s; PSS 5.88 GB peak during load, ≤ 4.23 GB after [F-023] [F-771] [F-772]. - The A15 and X6 ratios are not comparable in level (different judges) (group-A §4). - Deep tier on a phone: never run (as of 2026-10-24; OF-1 dropped by the author's 2026-10-21 decision, [D-147]). Projection 1.9–2.8 min per answer at budget 512 on a Pixel 8 Pro CPU, from third-party public rates and our arithmetic [F-766]. Earlier (10-09) estimate was 3.5–5 min at budget 1024 (group-B row 6). - Expert streaming fork on desktop CPU: 14–17 tok/s at ≈ 2.8 GiB RSS, judged-equivalent (0.625 vs 0.634, Δ −0.010, CI −0.042…+0.018, v0.1 dev, 1 seed, Opus). Desktop only; flash-bound projection ≈ 4 tok/s (UFS 3.1) / ≈ 6 tok/s (UFS 4.0) with compute [F-036] [F-036] [F-793]. Cross-built for Android arm64 on 2026-10-19; phone speed and memory never measured ([F-2118];; OF-2 dropped). - 8 GB phone (Pixel 8a, bench app, 2026-10-20/21): the 4B at 29–46 s per answer and one low-memory kill; the 1.7B at 13–16 s; this is chapter 11 [F-1418]. - Long sessions: no quality drop out to turn 15 (turns 6–15 in-session/standalone 1.136, 0.993–1.320; reference = BOAR's own standalone answer, Haiku L1, 80 agent-written questions); latency grows to ≈ 22 s at turns 14–15 vs ≈ 8 s standalone [F-030] [F-895]. **Key decisions:** [D-024] (repack-aware memory fit, Android only), [D-034] (KVFIX), [D-035] (no-network APK by construction), [D-046] / [D-053] ("no device exists", 10-07; reversed in practice 10-14), [D-119] (expert streaming supersedes earlier loader ideas on paper). **Turning point:** 2026-10-14 19:26, DEVICE-X6: the frozen build on a 12 GB-class phone gives the same judged quality as the desktop. #### Device parity and speed The frozen build (cut 7) on a 12 GB-class phone (MediaTek MT6897, 10.7 GiB usable, Android 16, airplane mode, no INTERNET permission) answered 72/72 items; on the 57 judged public v0.1 items, judged by Haiku L1, phone raw 0.529 (0.481–0.575) equals desktop 0.529, floor-adjusted 0.403 vs 0.402, paired −0.000 (−0.026…+0.028). Routing tier matched 60/60 and instant answers were byte-identical 16/16; model text was identical on 3/44 items (sampler differences). Earlier, cut 6 on an 8 GB phone scored 0.561 (0.507–0.617) vs desktop 0.558/0.560 under Opus judge.v1-med; the two phone studies are not comparable in level. Phone fast-tier p50 is 30.7 s (desktop 7.5 s on an M1 Pro; ×3.4); PSS peaks at 5.88 GB during load and stays ≤ 4.23 GB after. The deep tier never ran on a phone and was not delivered (author, 2026-10-21; [D-147]); two on-device engines are judged-equivalent to the desktop server on identical contexts (bmoe −0.012, −0.045…+0.020; expert streaming −0.010, −0.042…+0.018; v0.1, 1 seed, Opus), and a deep + 512 answer is projected at ≈ 1.9–2.8 min on a Pixel 8 Pro CPU from third-party rates. *Facts: [F-037] [F-036] [F-793] [F-766] [F-023] [F-771] [F-772] [F-773] [F-024] [F-775] [F-776] [F-777] [F-778] [F-779] [F-780] [F-781]* #### Multi-turn and lite tier Over 10 five-turn sessions and a 15-turn extension (80 agent-written questions, Haiku L1, reference = the system's own standalone answer, not the frontier), in-session answers score 1.136 (0.993–1.320) of standalone on turns 6–15; latency rises to ≈ 22 s at turns 14–15 vs ≈ 8 s. A 1.7B lite reader with X2b equals the plain 4B only on pack-matched items (−0.007, −0.031…+0.018); on the claim kit 1.7B − 4B = −0.111, and at identical contexts the 1.7B reads worse (−0.070, −0.094…−0.044, n = 97). An earlier internal summary ("the lite gap is reach, not reading") was retracted on this evidence. §6.8 reports the 1.7B as main engine from 2026-10-20. *Facts: [F-916] [F-030] [F-1081] [F-1082] [F-1279] [D-113]* ### 8 · The agent lab {#agent-lab} Span: 2026-10-01 → 10-24. Layers curious and tech: https://r4topunk.github.io/boar-research/research/agent-lab.md. **In one sentence.** One independent author ran this as a small research organisation made of AI agents: a coordinator that freezes rules and logs everything, researchers in lanes that propose but never measure, independent reviewers that decide wording, and an unattended loop; most ideas died for free, and the rules got stricter after every mistake. **Key facts:** - Lanes wrote technique cards: retrieval 19, models 22, evaluation/safety 23 [F-953]. - 101 closed arms: 54 closed at 0 judge calls, 10 at Haiku L1, 12 at L1 → Opus, 25 Opus directly [F-915] [F-873]. - Unattended loop 10-11 12:10 → 10-14 19:09: 133 ticks (28 idle); ≈ 12.0k Claude calls, of which ≈ 10.5k judge calls (Opus 4,851 + Haiku 5,660); mini 44.4 h + MacBook 18.1 h of generation; 1 dev lever; ≈ 31 avoidable machine-hours ([F-893] corrected-18, -22). - 29 numbered rule changes in 12 days [F-899]; every pass/fail rule hashed before the run: 65/65 verified on 10-10, 16 of 17 re-verified by an outside critic on 10-16 ([F-1169] corrected). - Incidents logged openly: a 10-hour run produced 45/120 empty answers without a smoke test [F-641]; the same 35B model file deleted twice by housekeeping ([F-756] corrected); judge authentication expired [F-228]. - Yield collapsed after the first week: 15 adopted improvements, all 10-04 → 10-08 [F-094]; 0 V-para floor-adjusted points per 1,804 judge calls in the 10-11/12 cycle ([F-1007] corrected). - Second phase (10-18 → 10-24): the author delegated seven open decisions to the coordinator, which adopted the recommended options [D-123] [D-124]; about 200 frozen-criterion arms ran, mostly 0-LLM replays and Haiku screens (`data/experiments.json` [X-198]…); a two-stage screen (GATE v2) set its bar from measured "do-nothing" arms, which already "fix" 23–31 % of problem items by chance [F-2073] [D-141]; a pipeline audit found 33 issues, 6 P0 [F-1908]; an independent direction review on 10-19 said "stop opening arms" and was followed only partly [D-129]; one overnight run generated 2,855 answers and judged none, leaving the choice of judge runs to the author [F-1726] [D-176]. **Key decisions:** [D-008] (lanes), [D-009] (freeze every criterion), [D-027] (no human review: independent reviewer agent + "AI-reviewed, not clinically reviewed"), [D-063] (fail-closed judge), [D-061] (Power block), [D-069] (unattended loop), [D-073] (relabel under-powered negatives), [D-094] (Rule 29 experiment economics), [D-095] (LOOP v2). **Turning point:** 2026-10-10 13:30, the R&D process review (10 agents, 76 findings): "measurement machine better than typical, pointed at the wrong target"; deliverables unbuilt [F-846]. #### 4 Discipline: frozen criteria, exposure ledger, fail-closed judging, and a judge funnel Each experiment ("arm") compares one change with a named control under a criterion written before the deciding run and frozen by sha256; verdicts may add dated addenda but not edit the criterion. 65/65 criterion hashes verified on 2026-10-10, and an outside critic re-verified 16 of 17 on 2026-10-16. From 2026-10-10 every criterion carries a Power block (expected Δ with its source, n, seeds, MDE); before that, 0 of 32 criteria frozen since 2026-10-07 stated one. From 2026-10-14 criteria also carry a Ceiling block (0-call upper bound; launch only if ≥ 2 × MDE) and a Budget block (smoke test, wall caps, kill rule). Judging is a funnel: deterministic 0-call gates (replays, counting) → Haiku L1 on a stratified random 20-item slice (futility screen only) → Opus confirmation and all claim numbers; a near-miss rule escalates any Haiku Δ within 0.01 of a threshold to Opus. Held-out sets are touched only by scripts that print aggregates; every touch is a ledger row (126 dated rows at audit). Negative results under-powered for the effect of interest are labelled "not detected at MDE x (n, seeds)" and kept in a registry with a reopen condition. **Second phase (2026-10-18 → 10-24).** About 200 further arms ran ([X-198]…), mostly 0-LLM replays and Haiku screens under a two-stage gate whose bar is set from measured null arms (GATE v2, [D-141]); seven open decisions were delegated by the author to the coordinator on 2026-10-18 [D-123]; one frozen FAIL was overridden with the spec reviewer's conditions (RULING R1, [D-144]); an overnight run generated answers and judged none, leaving judge runs to the author [D-176]. Who decided is labelled on every decision row. **Limitation stated here:** the planned 40-pair blind human label check was not done [F-211]. *Facts: [D-009] [F-1169] [D-061] [D-094] [D-057] [D-064] [D-073] [F-843] [F-185] [F-231]* ### 9 · Knowledge packs {#packs} Span: 2026-10-12 → 10-14. Layers curious and tech: https://r4topunk.github.io/boar-research/research/packs.md. **In one sentence.** When the bar stopped moving, the author pivoted to a product question: small, openly licensed offline libraries the user picks; one matching pack helps a little, many packs crowd each other out, and the real headroom is getting the one fact-bearing chunk into the prompt. **Key facts (180 pack self-test items, agent-written from pack text, dev; not a bar lever):** - 11 packs published with cards, licences and 20 public self-test questions each (Stack Exchange Q&A, first aid, MedlinePlus, a world factbook, Simple English Wikipedia; licences PD / CC0 / CC BY / CC BY-SA only) ([F-1028], -66; [D-080]). Count: 7 at 00:01 and 9 at 02:30 on 10-13, 11 by 19:34 (group-A §2 item 8). - One matching pack: +0.042 (+0.024…+0.062) Opus; +0.032 (+0.016…+0.050) Haiku [F-025]. - All packs as peers: fact article kept 68 → 46/180, quality −0.019 (−0.035…−0.003) Opus [F-026]. - Top-1 pack routing fails: 0.25 (vote) / 0.58–0.61 (4B zero-shot) vs a 0.85 bar [F-1026]. - Oracle: inject the known fact chunk: +0.095 (+0.070…+0.122) [F-027]. - X2b (append the retrieved article's neighbour chunks + budgeted dense candidates): fact present 35 → 102/180; +0.066 (+0.043…+0.091) Opus; inert on the bounty-shaped claim kit ([F-028], -89; [F-1076]). - The deep tier gains nothing from any pack: −0.002 (−0.020…+0.016) [F-029]. - V-para is not pack-matched: wave-1 packs cover 1/34 absent bounty-style items [F-091]. - All of English Wikipedia verifiably holds the missing fact for 9 of 34 absent bounty-style items: a lower bound, "at most about a quarter" ([F-1008]; group-B row 3). **Key decisions:** [D-078] (pivot to packs; conditional in-pack quality), [D-079] (pack-authoring workflow), [D-080] (format 2, licence policy, never NC), [D-084] (publish packs that pass the four-artefact check), [D-086] (oracle cell first), [D-087] (routing closed; rule + quota optional), [D-092] (X2b recommendation), [D-083] (default-eligible vs user-selected). **Reversal to show:** "facts are not in Wikipedia" (10-12 morning) withdrawn the same afternoon [D-076]. **Turning point:** 2026-10-13 23:11, ORACLE-ST: the ceiling is reach, not reading; it led directly to the only lever of the pack line. **Continued in chapters 11, 13 and 14:** with the 1.7B as main engine the pack line was re-measured on real development questions (no pack gain there, [F-2080]), re-framed as delivery (chapter 13) and became the author's closing thesis in a conditional form (chapter 14). The 4B-era numbers above stay as measured; they describe the 4B, not the 1.7B. #### Knowledge packs (pack-matched development items) On 180 pack-matched self-test items (agent-written from pack text; development data; one seed; deterministic 4B generation) one matching pack improves the 4B by +0.042 (+0.024…+0.062) under Opus and +0.032 (+0.016…+0.050) under Haiku. Installing nine packs as peers reduces fact-article survival from 68 to 46 of 180 and costs −0.019 (−0.035…−0.003). Top-1 routers reach 0.25–0.61 accuracy against a 0.85 bar. Injecting the known fact chunk yields +0.095 (+0.070…+0.122); HEADPASS-X2b raises fact-chunk presence from 35 to 102 of 180 and yields +0.066 (+0.043…+0.091) (Haiku +0.056), with +0.177 (+0.098…+0.263) on the 30 items where the chunk became newly present; on the bounty-shaped claim kit it is inert. The 35B gains nothing from packs (−0.002, −0.020…+0.016). These are conditional, in-pack results and are not evidence about the bar; questions written from the same text the retriever indexes favour retrieval. *Facts: [F-025] [F-1025] [F-029] [F-026] [F-1026] [F-027] [F-1027] [F-222] [F-028] [F-1063] [F-1064] [F-095] [F-1076]* ### 10 · Measuring the measurer {#measurer} Span: 2026-10-02 → 10-21. Layers curious and tech: https://r4topunk.github.io/boar-research/research/measurer.md. **In one sentence.** Claude wrote the questions, wrote the references and judged them, so the project measured its own instrument: judge noise is tiny, seed noise is not, and a judge from another company scores everything about 0.10 higher while keeping the same ranking and a near-zero win-rate. **Key facts:** - Second Claude judge (Fable 5.1): same winner 453/456 [F-1142]. - Haiku 5.5-low vs Opus on v0.1: r 0.922, κ 0.651, ratio −0.069 (lead with these; "same winner 54/57" is near-uninformative because the reference wins ≈ 100 % of items) ([F-996] OVERCLAIM note). - Local open-weights judge (Gemma 4 26B): +0.11 lenient, κ 0.32/0.55, same ranking ([F-164], -47). - Opus test-retest: σ ≈ 0.003 raw; per-item r 0.96; winner 51/51 [F-019]. - Seed noise: SD 0.020 raw / 0.025 floor on the deep tier; two seeds differ −0.039 (−0.072…−0.011) ([F-012], -34). - Reference length: shortening references to ≤ 80 words moves H floor-adj 0.412 → 0.514 (s42); ≈ 17 % of the gap is length, ≈ 83 % substance [F-467]. - Correctness alone: H floor-adj 0.660 (0.608–0.712) on V-para; completeness 0.301, usefulness 0.341 [F-284]. - Cross-family judge (post-hoc, 408 + 660 calls, same rubric, orders and blinding): V-para 4B raw 0.497 / floor 0.361; H raw 0.667 / floor 0.577 (0.519–0.632); win-rate 0.00–0.01. v0.2: 4B 0.617 / 0.517 (2 seeds); H 0.714 / 0.639 (1 seed); deep − 4B gap +0.097 vs Claude +0.096; win-rate 2.7 % / 6.4 % vs 0 % [F-017] [F-017] [F-017] [F-018] [F-018] [F-490] [F-1208] [F-232]. - Outside critique: numbers real (15/17 re-derive), internal summaries overstated; 7 re-wordings applied [F-227] [D-113]. - Later calibration (10-20/21, dev): Haiku and Opus agree on confident errors (κ 0.75) far better than on "correct" (κ 0.49; 0.22 on warned answers); Opus finds 98 correct answers Haiku misses vs 10 the other way, so Haiku-only correctness rates are low-biased [F-2083] [F-1432]. judge.v1 gives a bare refusal U 0.058 (0.037–0.081), not zero [F-2081]: the reason the second ruler adds a stage-1 zero (chapter 12). **Key decisions:** / (hold the cross-family pilot; done later with OpenAI gpt-5.6-sol via the Codex CLI), [D-049] (Haiku as L1), [D-050] (open judge for ranking only), [D-057] (near-miss rule), [D-072] (common-mode CI), [D-122] / [D-115] (lower judge is the headline; family named next to every ratio). **Turning point:** 2026-10-16 19:01, the cross-family judge: the deep tier crosses 0.50 floor-adjusted only under the other family, and the win-rate does not move. #### 7.1 One family wrote, referenced and judged: what changes under another Items (v0.2, V-para), references and the judge all come from the Claude family. We ran four checks. A second Claude judge (Fable 5.1) agrees with Opus on 453/456 verdicts. Haiku 5.5-low tracks Opus on v0.1 with r = 0.922, κ = 0.651 and a −0.069 level offset (agreement on the winner, 54/57, is near-uninformative because the reference wins almost every item) and compresses paired gaps to 0.70–0.95× of Opus's on pack arms and as low as 0.25× on one 4B-vs-deep pair. A local Gemma 4 26B judge is +0.11 lenient (κ 0.32/0.55) but preserves the deep − 4B ranking (+0.026 vs +0.029). Post hoc, the author re-judged existing traces with an OpenAI model using the same rubric, orders and blinding (408 calls on V-para, 660 on v0.2). It scores every configuration higher: on V-para the 4B rises from 0.413 / 0.261 to 0.497 / 0.361 and the 512 configuration from 0.553 / 0.436 to 0.667 / 0.577 (0.519–0.632); on v0.2 the 4B rises to 0.617 / 0.517 (two seeds) and the deep configuration to 0.714 / 0.639 (0.579–0.695) (one seed). The deep − 4B gap is preserved (V-para +0.171 vs +0.139; v0.2 +0.097 vs +0.096), per-item Spearman is 0.88–0.97, and win-rate stays near zero (V-para 0.00–0.01; v0.2 2.7 % and 6.4 % vs 0 %). Per the analysis plan, when two families differ by more than 0.05 raw the lower is the headline; we report the Claude numbers as headline and the cross-family numbers beside them. **Caveat line under the figure:** "Cross-family runs were not pre-registered; their reports live with the held-out data and were not opened by the authors of this page; CIs for the 4B floor-adjusted cells were not available." *Facts: [F-1142] [F-996] [F-1153] [F-1187] [F-164] [F-583] [F-017] [F-018] [F-490] [F-1208] [F-232]* #### Noise budget Re-judging unchanged answers with Opus gives σ_judge ≈ 0.003 raw (per-item r = 0.96; winner 51/51). Re-generating answers is the dominant noise: on the deep tier, seeds 42/43/44 give raw 0.533 / 0.572 / 0.544 (SD 0.020; floor-adjusted SD 0.025), and two seeds of the same configuration differ by −0.039 (−0.072…−0.011), a CI excluding zero. The 4B on V-para is seed-stable (identical ratios across seeds; per-item r = 0.81), and the desktop 4B path is byte-deterministic on a fixed host (20/20). A single 20-item Haiku pass carries ±0.03–0.04 of judge noise, about the size of the decision threshold, so slices screen for futility only; "hardest-20" slices inflated deltas 3–10× and were retired. The house reading rule is that a one-seed deep-tier Δ on V-para below 0.05 is seed noise (raised from 0.03 on 2026-10-16). *Facts: [F-019] [F-012] [F-020] [F-618] [F-463] [F-476] [F-1166] [F-184] [F-199] [F-1195] [D-058]* #### Multiplicity and selection Across ≈ 60 judged comparisons, ≈ 1.5 false "CI low > 0" results are expected at α = 0.05; the large levers survive a Bonferroni-style correction (X2b z ≈ 5.5, LITE5 z ≈ 6, thinking z ≈ 4.2). The larger risk is selection on the same development sets (winner's curse on the deep configuration; levers designed on the 180 self-tests), which is why development and held-out readings are separated throughout. *Facts: [F-1079] [F-1078]* #### Two judges on labels, refusals and correctness (2026-10-20/21) On double-labelled development units Haiku and Opus agree on confident errors (κ 0.75) far better than on "correct" (κ 0.49; 0.22 on warned answers); Opus finds 98 correct answers that Haiku misses vs 10 the other way, a +0.02 U shift [F-2083]. A Haiku confident-error count moves by about ± 1.3 per 100 on retest of identical answers. judge.v1 gives non-answers a positive score (bare refusal 0.058, hedge 0.053, fluent no-content 0.016; [F-2081]), and attempts to fix this inside the rubric flipped 5–7 of 40 confident-error verdicts [F-1955], hence the stage-1 zero of §3.6. Reference draws matter little: two uncapped Opus draws move U by ≤ 0.015 [F-2114]. Same-family judge lift in the literature is 3.4–8.4 points for open-model families and untested for Claude and GPT [F-2115]. ### 11 · A smaller engine for 8 GB phones {#small-engine} Span: 2026-10-18 → 10-21. Layers curious and tech: https://r4topunk.github.io/boar-research/research/small-engine.md. **In one sentence.** On a real 8 GB phone the 4B was too slow and once killed by the operating system, so on 2026-10-20 the author made Qwen3-1.7B the main 8 GB engine, for latency and memory and explicitly not for quality. Two days of screens found no small model, prompt, decoder or gate that closes the 1.7B's reading gap at the same phone time, and the product rule became "refuse by default when no source is kept, with a warning line on what is shown". **Key facts (dev data throughout; bench app or desktop research engine, never the submitted app):** - Why no 8 GB lever could move the bounty-style number: on V-para the 4B's gap of 0.586 ratio points splits into wrong-from-memory 0.309 (23 items), absent knowledge 0.239 (24), thin 0.038 (4), refusals 0 (Opus judge.v1, seed 42, single-seed labels ± 0.05; [F-1406]). The fact is in the installed kit for 6 of 51 items and in any pack on disk for 10 of 51 [F-1407]. Estimated ceiling of all 8 GB levers on V-para: about +0.01–0.02 raw [F-1411]. - Honest prompt for the app's longer style: honesty clauses cut the audited wrong-claim rate 0.231 → 0.062 (paired −0.169, CI −0.281…−0.073; Claude Opus sentence audit, 15 agent-written chat turns per arm, 407 claims, dev) at no judged-ratio cost [F-1394] [F-1397]. - The pivot: on a Pixel 8a (8 GB) the 4B decoded 3.6–6 tok/s, answered in 29–46 s and was killed once at 4.4 GB RSS; the 1.7B decoded 13–16 tok/s and answered in 13–16 s (9 public dev questions, before batched lookups; [F-1418]). The author made the 1.7B the main 8 GB engine and the 4B an optional "more complete" tier ([D-134] / [F-1386]). - Batched pack lookups (937 → 48 SQL calls per retrieval, retrieval byte-identical 390/390) cut the 1.7B's time to first token on the phone from 9.12 to 5.55 s (n 9; [F-1422], [F-1423]). - 73-question phone runs (bench app, Pixel 8a, v0.1 dev, one run each): 1.7B p50 4.2 s over all items, PSS 3.39 GB; Gemma 4 E2B 11.8 s, PSS 4.52 GB; 4B 26.0 s complete (older engine), PSS peak 4.76–4.92 GB [F-1565] [F-2078]. **Caveat that travels with the 4.2 s:** it covers deterministic answers and declines (the 1.7B declined 35 % of v0.1); its model-path p50 is about 12 s; never set it beside the 13–16 s of the 9-question runs. - The cost is reading. Under Opus (judge.v1 floor-adjusted ratio "U", v0.1 51 dev, one seed per rung): 4B − 1.7B +0.109 (+0.063…+0.160); 4B − Gemma E2B +0.098 (+0.054…+0.146); 1.7B − closed-book 1.5B +0.090 (+0.009…+0.170), which under Haiku is +0.038 (−0.023…+0.098), so that last step is judge-dependent [F-2077]. At identical pack contexts (54 pack self-tests, Haiku raw): 4B 0.650, LFM2.5-8B-A1B 0.554, 1.7B 0.529 [F-1401]. - Screens that did not close it (GATE v2, two-stage; [F-2072], [F-2117]): six prompt, decoding and confidence levers (fixes 1–4 of 21–23 confident errors, bar 6; token-probability AUC 0.530); decoder-level grounding [D-159]; Ministral-3-3B with a citation prompt passed stage 2 (Opus +0.047) but at about 3× phone time and below the 4B (−0.037 Opus), closed as dominated [D-189] [F-1933]. Gemma 4 E2B: Haiku +0.028, a FAIL by 0.002; Opus +0.031 (+0.005…+0.058), a GO by 0.001 [F-2076]; on the phone 11.8 s vs 4.2 s and +1.1 GB; kept as an opt-in proposal only [D-143]. - Honesty surface: the author set "no source → refuse by default, with a 'show anyway' control" [D-138] [D-149]. A groundedness gate cut visible confident errors 7 → 3 of 177 under an Opus re-judge (Haiku 10 → 4), false refusals 0 of 14, at about +9.3 s per answer projected for the Pixel 8a (5.1 s measured on a desktop); it is not in any app [F-1430] [F-2074]. Revealed answers behind "show anyway" were 11 correct to 16 confident errors in an Opus census [F-2084], so the button is a liability: it sits behind a second confirmation and never on health [F-1633]. The warning line's precision is 0.39–0.40 and recall 0.65–0.67 on the 1.7B (Haiku / Opus; [F-2085]). No drift detected over five turns [F-1957]. AI-reviewed, not clinically reviewed. **Key decisions:** [D-123] / [D-124] (delegated decisions D1–D7: no v0.3, site only as v1.0 at closure), [D-129] (direction review: consolidate), [D-134] (1.7B main engine), [D-141] (GATE v2), [D-149] (refuse by default), [D-159] (decoder grounding closed), [D-186] (keep Q4_K_M), [D-187] (1.7B on latency and memory, not quality; model screens parked), [D-189] (Ministral closed). **Reversals to show:** the 10-20 morning proposal "answer under a warning line" was replaced the same evening by refuse-by-default ([D-135] → [D-149]); the 10-19 release of a safety-failing patch was reversed the same day [D-128]. **Turning point:** 2026-10-20 ~11:50, the author: "Qwen3-1.7B becomes the main engine". #### 6.8 Latency and memory over quality: the 1.7B as main engine A V-para diagnosis decomposed the 4B's 0.586 ratio-point gap into wrong-from-memory 0.309 (23 items), absent knowledge 0.239 (24), thin 0.038 (4) and refusals 0 (Opus, seed 42, labels ± 0.05), with the fact present in the installed kit for 6 of 51 items; the estimated ceiling of all 8 GB levers on V-para was ≈ +0.01–0.02 raw [F-1406] [F-1407] [F-1411]. On a Pixel 8a the 4B answered in 29–46 s and was killed once at 4.4 GB RSS, the 1.7B in 13–16 s (9 public dev questions; [F-1418]); the author made the 1.7B the main 8 GB engine on 2026-10-20 [D-134]. Under Opus on CV5 51 (judge.v1 floor-adjusted ratio of sums, one seed per rung) the 4B leads the 1.7B by +0.109 (+0.063…+0.160) and Gemma 4 E2B by +0.098 (+0.054…+0.146); the 1.7B's lead over a closed-book 1.5B is +0.090 (+0.009…+0.170) under Opus and +0.038 (−0.023…+0.098) under Haiku, i.e. judge-dependent [F-2077]. Two-stage screens (GATE v2; the bar = the largest measured null-arm net + 2, after null arms "fixed" 23–31 % of problem items; [F-2073]) closed prompts, decoding, token-confidence gates (AUC 0.530), decoder grounding, fine-tuned readers and small-model swaps; Gemma 4 E2B's stage-2 verdict flips with the judge (Haiku −0.002 below the bar, Opus +0.001 above; [F-2076]) at 2.8× phone time. A groundedness gate cut visible confident errors 7 → 3 of 177 under an Opus re-judge (Haiku 10 → 4), false refusals 0/14, at ≈ +9.3 s per answer projected on the phone [F-1430]. The reveal-anyway control returned 11 correct to 16 confident errors in an Opus census [F-2084]. No drift was detected over five turns [F-1957]. ### 12 · A second ruler (bench v0.4) {#new-ruler} Span: 2026-10-19 → 10-24. Layers curious and tech: https://r4topunk.github.io/boar-research/research/new-ruler.md. **In one sentence.** The first ruler scored a desktop engine against Claude-written references with a Claude judge on Claude-written questions. For the submitted app the project wrote a second one: 75 new questions frozen on 2026-10-20 before any system answered them, written half by each judge family, scored as U-total with a zero for non-answers, two judge families and a split by whether the packs hold the answer. As of 2026-10-25 it has references from both families, a desktop-twin run of the submitted app's pre-rename engine that the author published as history (0 of 71 wins; lower family below the 0.50 bar), and an on-phone run in progress that will carry the headline. **Key facts:** - Design (10-19): a public ruler with a true-zero utility, uncapped two-family web references, items written outside the judge family where possible, and the submitted app as the scored system ([D-131] / [F-1383]). - Bench v0.4: 75 items (71 headline + 4 anchors taken from the example questions in Vitalik Buterin's post that inspired the bounty, scored outside the headline), frozen 2026-10-20 16:45, sha f9ad2f2f, before any system answered [F-2070] [F-1353]. Candidate writers: Claude 36, OpenAI gpt-5.6-sol 34 [F-1416]. 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both). It is **not** written outside the Claude family: half is Claude-written, a stated deviation [G-ruler-v04]. Its item text and references are not public [D-195]. - Metric: U-total, a ratio of sums of (rubric mean − 1) for the app over the reference, so the rubric floor maps to 0; health items weigh 0.5 with weight-1 printed beside ([D-137]; the weighting was decided after the items existed, [F-1389]); verdict = the lower of the two families, "meets the bar" only if CI low > 0.50 with all gates passed [G-u-total]. - Why a stage-1 zero: judge.v1 gives a bare refusal U 0.058 (0.037–0.081), a hedge 0.053, a fluent no-content answer 0.016 (Haiku, 30 v0.1 dev ids; [F-2081]). A binary-checklist judge v2 failed its retest gate (CE retest 0.966 vs a needed 0.974; [F-1431]). RULING R1 kept judge.v1 unchanged and added DECLINE2STAGE, a classifier that zeroes detected non-answers; it was a coordinator override of a frozen FAIL, accepted with conditions by the spec reviewer ([D-144] / [F-1529]). - Spec r3 (sha 0fac2286, 20:48) and r4 (sha 82772e33, 21:18) were frozen on 2026-10-21 before any app answered a v0.4 item [F-2109]. Verdict arm A∪: the app with all six persona packs installed at once [D-139] [D-156]. - References: Claude half 75/75 frozen before any app answer (uncapped, median 703 words, median 6 sources) [F-2110]; OpenAI gpt-5.6-sol half 75/75 on 2026-10-24 20:05, 0 hard failures [F-2212]. - Information available (counts only; label lists quarantined): the A∪ packs hold the answer for 19 of 71 headline items, partly for 25, not at all for 27 (strict 15 / 23 / 33) [F-2111]. 7 current-city items carry 10.7 % of headline weight and no app could answer them [F-2113] [F-1960]. - Exposure: on 2026-10-24 two engines answered v0.4 for the first time (desktop twins, Claude half only, stage 1 not run). Those numbers stay internal (one of the two engines is the native engine branch, whose numbers stay with the app maintainer; AD-15 (b)) and are dev/parity evidence, never a verdict; any later engine change counts as developed after exposure [F-2108]. - The scored system is the submitted APK on a phone [D-151] [D-191]. As of 2026-10-24 20:34 no phone run on v0.4 exists. - Final run (RULER-FINAL, 2026-10-24 20:34): BoarLite's final engine b4ef94f3 on its desktop twin, 71 headline items, one seed (the spec asks for two; the author's choice), each judge family against its own references, stage 1 not run [F-2214] [D-204]. The ruler is now exposed to three engines. Published by the author's decision of 2026-10-25 as history: 0 of 71 wins under both families (one gpt-5.6-sol tie); U-total (desktop twin of engine b4ef94f3) Claude Opus 0.164 (0.123–0.208), OpenAI gpt-5.6-sol 0.245 (0.194–0.300); the lower family, 0.164 (0.123–0.206), is below the 0.50 bar [F-2215] [X-405]. Information-available split, Claude family: 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27): with the information present, the 1.7B did not deliver it [F-2217]. The paired delta against the native engine branch stays internal, because it prints that branch's number (U44). They are numbers for b4ef94f3: the shipped app, LabRat 7788924a, adds the Rat persona line to the grounding prompt after this run [F-2225]. The on-phone re-run of v0.4 on LabRat (DEVICE-FINAL, three seeds under Claude Opus, seed 42 under gpt-5.6-sol) started on 2026-10-25 under a criterion frozen before generation ([F-2227]; OF-8); no result is in this record. **Key decisions:** [D-131], [D-132], [D-133] (freeze; one item dropped after both families contradicted it), [D-137], [D-138], [D-139], [D-144] (RULING R1), [D-151], [D-156], [D-157], [D-158], [D-195]. **Supersessions to show:** the v0.3 path to an unexposed number (D5, 2026-10-18, [F-1374]) → v0.4, which is exposed and half Claude-written; "the ruler was frozen on 10-20" → the *item set* was frozen on 10-20, the first frozen spec is r3 on 10-21 [F-1628]. **Turning point:** 2026-10-20 16:45: the v0.4 item set is frozen before any system has seen it. #### 3.6 A second instrument for the submitted app Because the first instrument scored a desktop engine against Claude-written references on Claude-written items, a second ruler was specified for the submitted app [D-131]. Bench v0.4 has 75 items (71 headline; 4 anchors taken from the example questions in Vitalik Buterin's post that inspired the bounty, outside the headline), drafted by Claude (36 candidates) and OpenAI gpt-5.6-sol (34), cross-fact-checked (329 checklist points supported on the open web by at least one family with none contradicting), and frozen on 2026-10-20 before any system answered (sha f9ad2f2f) [F-2070] [F-1416]. The headline statistic U-total is a ratio of sums of rubric mass above the floor (mean − 1) against uncapped frontier-plus-web references from both families, with health items at weight 0.5 (weight-1 beside; decided after the items existed, [F-1389]) and a stage-1 classifier that zeroes detected non-answers (DECLINE2STAGE), because judge.v1 scores a bare refusal at U 0.058 (0.037–0.081) [F-2081]. A binary-checklist judge v2 failed its pre-set retest gate [F-1431]; keeping judge.v1 with the stage-1 zero (RULING R1) overrode a frozen FAIL and was accepted with conditions by the spec reviewer [D-144]. Spec r3 (sha 0fac2286) and r4 (sha 82772e33, adding an information-available split, FACT-AVAIL, and ORACLE / ORACLE-0 rows) were frozen on 2026-10-21 before any app answer [F-2109]. The verdict arm installs all six persona packs (A∪); the verdict is the lower family per bootstrap draw with CI low > 0.50 and all gates. FACT-AVAIL labels: AVAILABLE 19 / PARTIAL 25 / NONE 27 of 71 (strict 15 / 23 / 33), lists quarantined [F-2111]. As of 2026-10-24 20:34: references 75/75 for both families (Claude median 703 words; gpt-5.6-sol completed 20:05, 0 hard failures, [F-2212]); the set counts as exposed from 2026-10-24 [F-2108] and has now been answered by three engines on desktop twins. The final run (RULER-FINAL, [F-2214], [D-204]) scored BoarLite's engine b4ef94f3 with one seed (spec: two), each judge family against its own references (not the two-family mean), stage 1 not run, no errata; the spec labels desktop-twin numbers dev/parity evidence. Published by the author on 2026-10-25 as history: win-rate 0 of 71 under both families (one gpt-5.6-sol tie); U-total (desktop twin of engine b4ef94f3) 0.164 (0.123–0.208) Claude Opus, 0.245 (0.194–0.300) gpt-5.6-sol; minimum over families 0.164 (0.123–0.206), below the bar by the pre-registered rule; FACT-AVAIL (Claude) AVAILABLE 0.164 (n 19), PARTIAL 0.174 (n 25), NONE 0.155 (n 27), so the information's presence did not raise U-total [F-2215] [F-2217]. The paired delta against the native engine branch is withheld (it carries that branch's absolute; the app maintainer's numbers stay internal). The engine it measured, b4ef94f3, precedes the rename to LabRat: 7788924a changes the grounding prompt's identity line ("You are Rat…"), so the RULER-FINAL number is not attributed to LabRat without that caveat [F-2225] (the ruler run predates the Rat persona line). DEVICE-FINAL (criterion frozen 2026-10-25 03:3x, [F-2227]) re-runs v0.4 on the phone with LabRat 7788924a, three seeds under Claude Opus and seed 42 under gpt-5.6-sol, with a phone-vs-desktop parity read against b4ef94f3 that is not a verdict; running, no results in this record. ### 13 · Delivery, not coverage {#delivery} Span: 2026-10-19 → 10-23. Layers curious and tech: https://r4topunk.github.io/boar-research/research/delivery.md. **In one sentence.** With the 1.7B as main engine, the question became whether the one passage that holds the answer reaches the model. On questions written from the packs it often does, and answers jump; on real development questions the pack gain disappears, and almost every undelivered fact already sits in an installed pack. Search fixes delivered more facts; no answer-level gain was established, and one fix was withdrawn as unsafe. **Key facts (all dev; "U" = judge.v1 floor-adjusted ratio; Haiku unless named):** - Self-tests vs real items: on 60 topical pack self-tests the pack gain sits on items where the pack adds the key fact, +0.139 (+0.075…+0.201), n 27; the same cell on real development items is −0.010 (−0.077…+0.050), n 10 [F-2080]. The self-test gain is not a word-overlap artefact (reweighted to real overlap +0.064, +0.035…+0.095; [F-1965]). - Question form decides retrieval (48 pack facts × 9 phrasings, agent-written, 0-LLM replay): the gold chunk is kept for 50 % of base questions, 83 % of yes/no, 25 % of situational, 10 % of entity-free, 8 % of bare follow-ups [F-2092]. When the reference chunk is kept, the 1.7B scores U 0.876 (n 43, 2 confident errors) vs 0.185 (n 147, 75 confident errors), a between-item comparison on pack-written items [F-2093]. - The author, 10-22: "packs are the strategy"; small focused packs, full Wikipedia as a negative control [D-161] [D-162]. - A stricter count: from 10-22 a fact is "delivered" only when it sits inside one kept chunk; article-level counting had overstated delivery by 8–13 facts [D-165] [F-2095]. - Search fixes on MAIN 92 keyed items, chunk level: none 21 → global-IDF merge 28 → SEFIX-124 42 → SEFIX-ALL 47 [F-2095]. SEFIX-ALL's fourth fix (F3, a health no-source fallback) fired on 21 of 65 safety probes and unsafe answers rose 9 → 13 (blinded agent reading, one seed; AI-reviewed, not clinically reviewed); it was withdrawn [D-164] [F-2097]. The safe set SEFIX-124 gives +0.013 (−0.008…+0.035) on all 97 (two seeds): not a lever, adopted for delivery only [F-2025]. - Between items, delivery does not separate answers: delivered 0.420 vs not 0.419 (+0.001, −0.071…+0.069); within the same items, +0.044 (+0.009…+0.082, n 42, two seeds), and an Opus check on 28 delivered general questions +0.013, inside noise [F-2098]. - Where the undelivered facts are: 44 of 50 already sit in the default 13-pack kit (K13); new packs reach at most about one fact each [F-2100], so no new packs were built [D-173]. - Reading cost: a compact index inside each pack (W78) cut the data read per question from 125.09 to 21.26 MB and the cold first stage from 620 to 92 ms, with 42/42 facts kept (desktop replay; phone unmeasured) [F-2103]. **Key decisions:** [D-127], [D-160], [D-161], [D-162], [D-164], [D-165], [D-172], [D-173], [D-193], [D-194]. **Corrections to show:** F1b's "45 delivered" is 32 at chunk level; SEFIX-ALL's "55" is 47 [F-2095]; the Opus +0.173 "where the fact arrives" (n 22, selected slice) came from the run that included the withdrawn F3 [F-1593] [F-2096]. **Turning point:** 2026-10-22, the author's "packs are the strategy", followed the same day by the chunk-level rule that made the target measurable. #### 6.9 Self-test gains do not transfer; delivery does not separate answers On 60 topical pack self-tests the pack gain concentrates on items where the pack adds the key fact (+0.139, +0.075…+0.201, n 27); on real development items the same cell is −0.010 (−0.077…+0.050, n 10), and the self-test gain is not a lexical-overlap artefact (+0.064 after reweighting to the real overlap distribution) [F-2080] [F-1965]. Retrieval depends on question form (hit@kept 83 % yes/no, 50 % base, 25 % situational, 10 % entity-free, 8 % bare follow-ups; [F-2092]). From 2026-10-22 delivery is counted at chunk level (article-level counts overstated it by 8–13; [D-165]). On MAIN 92 keyed items, delivered facts rose 21 → 28 (global-IDF merge) → 42 (SEFIX-124) → 47 (with F3, withdrawn after unsafe answers rose 9 → 13 on 65 probes) [F-2095] [F-2097]; the safe set changes U by +0.013 (−0.008…+0.035; Haiku, two seeds) [F-2025]. Between items, delivered vs not scores 0.420 vs 0.419 (+0.001, −0.071…+0.069); within items +0.044 (+0.009…+0.082, n 42, Haiku), Opus on 28 delivered general items +0.013 [F-2098]. 44 of 50 undelivered facts already sit in the installed kit [F-2100]. ### 14 · Closing the research: what a small model needs {#closure} Span: 2026-10-23 → 10-24. Layers curious and tech: https://r4topunk.github.io/boar-research/research/closure.md. **In one sentence.** On 2026-10-23 the author closed the research lines; on 2026-10-25 he stated the thesis the record presents: **small models need external knowledge, and the project, having learned that, built a harness around a small model's real capabilities, tuned for the app user's experience** (a health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour) [F-2236] [D-215]. The closing tests are its evidence and its honest limits: with the matching pack the 1.7B is right on 48 % of pack-written self-tests vs 26 % without, and on 65 % when the item's own article reaches the model vs 28 % otherwise; but the article arrives only 54 % of the time, no pack reached the 0.75 bar fixed in advance, and the gain did not transfer to real development items. The sentence the author first wrote on 2026-10-23, "right whenever the content is in a pack", is kept as the hypothesis those tests checked: they support it only in that conditional form. The afternoon turned to content (reshaping packs), the night to validating one closed engine, and the next day's judge kept the shortest answer style. **Key facts (outcome-label shares unless named; Claude Haiku 5.5-low labels; Qwen3-1.7B, Succinct; seed 42; desktop twin, no phone; dev):** - Closure: the author closed the research lines; the rest is acceptance tests on the app's desktop twin [F-1653] [D-167] [D-169]. - Thesis, first form (2026-10-23 12:19): "right whenever the content is in a knowledge pack, and the app built to answer well under that premise" ([D-170] / [F-1690]); the closing tests support it only in the conditional form [G-conditional-thesis]. Superseded as the thesis statement on 2026-10-25 ([D-215] / [F-2236]): small models need external knowledge; a harness built around a small model's real capabilities, tuned for the app user. Its limits include the real-item split: the self-test fact-added cell +0.139 (+0.075…+0.201, n 27) vs real development items −0.010 (−0.077…+0.050, n 10) (Haiku judge.v1 U, one seed; [F-2080]), and the app's desktop-twin ruler reading (lower family 0.164, below the bar; [F-2215]). - PACKPROOF (180 pack-written self-tests of 9 packs): correct-any 0.48 with the pack vs 0.26 without, +0.23 (+0.16…+0.31) [F-1704]; 0.65 when the item's own article is kept (n 98) vs 0.28 [F-1705]; the own article is kept for 54 % and cited for 28 % (article-level; [F-1706]); 0 of 9 packs reach 0.75 (best 0.60, worst 0.25; [F-1707]). Without the pack: wrong and confident 0.17, wrong under the warning line 0.45, honest "unknown" 0.12 [F-1708]. - UXMATRIX (60 dev items): Succinct beats Detailed with and without the pack (correct-any ON 0.35 vs about 0.27; [F-1709], [F-1710]); with the pack off the model never says "I don't know" itself: every honest "unknown" is an engine decline, and the warning line shows on 75 % of answers [F-1711]. - Pack and search levers closed the same day (0-LLM replays): undelivered facts are a search problem, 44 of 50 already in K13 [F-2100]; dropping the Wikipedia packs loses MAIN facts 42 → 33, so they stay [F-1714] [D-172]; an oracle question rewrite would deliver only +9 facts (bar +10) [F-1715]; warning cited-but-unsupported answers would catch 9 of 26 wrong at a cost of 5 of 37 correct, NO [F-1712]. - CONTENT-FIRST (author, 15:3x): no model change; make the right chunk reach the 1.7B [D-174]. Shaped factbook (short sentences that name the country, no new facts): fact chunk kept 7 → 15 of 20; correct 5 → 10 of 20 (two seeds, Haiku); Opus judge.v1 U 0.316 → 0.436 (+0.121, +0.030…+0.221, two seeds); NOT-CONFIRMED by its frozen rule because of one new warned confident error [F-2104]. The 0.436 here is a factbook self-test U and has nothing to do with the deep tier's 0.436 on V-para. - ENGINE-RC (night, one closed tree): every 0-LLM suite passed (identity 97/97 and 180/180; routes 103/110; the one letter-FAIL pre-existed); data read per question 125.09 → 21.26 MB; a false "Not found in this phone's offline library" line on 37 of 97 answers with kept passages fell to 0; 2,855 answers generated, 0 judge calls overnight [F-2105] [F-1722] [F-1725] [F-1726]. - Validation judge (2026-10-24, the page's numbers): Succinct correct-any 0.43 with the pack (right with source shown 0.30, wrong-unwarned 0.12), 0.22 without (wrong-unwarned 0.10); longer styles (one 4–5-sentence paragraph, two paragraphs) were right as often (0.42) but doubled wrong-unwarned answers (0.23, 0.25) [F-1727] [F-1728] [F-2106]. Follow-ups: carrying the subject brings the turn-1 source back (0.083 → 0.708) without changing accuracy (0.458 both) [F-1729] [F-2107]. The 10-23 → 10-24 Succinct shift (0.35 → 0.43) has no isolated cause in the record (engine changed in between; n 60, one seed). **Key decisions:** [D-215] (thesis restated, 2026-10-25), [D-167], [D-168] (adopted for delivery: SEFIX-124, W78, Succinct for models ≤ 2 GB, Deep Research hidden), [D-169], [D-170], [D-171], [D-172], [D-173], [D-174], [D-175] (shaped factbook stays user-selected, unpublished), [D-176], [D-177] (wider append budget and best retrieval config stay out: dose/number probes changed), [D-179], [D-180]. **Turning point:** 2026-10-23 11:53, the author closes the research lines; at 12:19 a first thesis is stated, and by 14:2x the tests have reduced it to its conditional form; on 2026-10-25 the author restates the thesis as what was learned and built, with those numbers as its evidence and limits. #### 6.10 Small models need external knowledge: what the closing tests support The author closed the research lines on 2026-10-23 with the hypothesis "right whenever the content is in a pack" [D-167] [D-170]; on 2026-10-25 he stated the thesis as a design statement: small models need external knowledge, and the project built a harness around a small model's real capabilities, tuned for the app user's experience [F-2236] [D-215]. The numbers below are its evidence and its limits; they do not establish a quality level (U41–U43). Closing tests ran on the desktop twin with outcome labels (Claude Haiku 5.5-low; §2.13 of the claims page): on 180 pack-written self-tests of 9 packs (1.7B Succinct, seed 42), correct-any is 0.48 with the pack vs 0.26 without (+0.23, +0.16…+0.31); 0.65 when the item's own article is kept (n 98) vs 0.28; the own article is kept on 54 % and cited on 28 % (article level); 0 of 9 packs reach the 0.75 bar fixed in advance ([F-1704]…[F-1708]). The hypothesis therefore holds only in the conditional form, and on real development items the pack gain does not transfer (fact-added cell −0.010, −0.077…+0.050, n 10, vs +0.139 on self-tests; [F-2080]). Pack shaping (factbook) doubled delivery and correct answers on 20 self-tests and moved Opus U 0.316 → 0.436 (+0.121, +0.030…+0.221), NOT-CONFIRMED by one new warned confident error [F-2104]. The closed engine passed every 0-LLM validation suite and cut data read per question 125.09 → 21.26 MB [F-2105] [F-1722]. The 2026-10-24 validation judge kept Succinct: correct-any 0.43 vs 0.42 for longer styles, wrong-unwarned 0.12 vs 0.23–0.25 (UXMATRIX 60, seed 42) [F-1727] [F-1728]; the Succinct reading moved from 0.35 on 10-23 to 0.43 on 10-24 on the same items after engine changes, with no isolated cause. Subject carry restored the turn-1 source on follow-ups (0.083 → 0.708) without changing accuracy (0.458) [F-1729]. ### 15 · One simple app: LabRat {#submission} Span: 2026-10-15 → 10-25. Layers curious and tech: https://r4topunk.github.io/boar-research/research/submission.md. **In one sentence.** Which app is submitted changed twice: the app maintainer's native app (decided 10-15, reaffirmed 10-21), then on 2026-10-24 BoarLite, a closed, offline, Succinct-only chat app of the author's own, renamed LabRat on 2026-10-25 ("LabRat (formerly BoarLite)"; below, BoarLite for events before the rename). In one evening (2026-10-24 17:39 → 20:34) BoarLite went from a decision to a final candidate (b4ef94f3): a health gate that shows fixed safe text when no health source is kept, one fixed line under every answer, factbook v1, conversions computed in code, and the bench search ported in a mode that skips health questions. It runs offline on a Pixel 8a. The search port was merged although its frozen criterion failed by the letter, because the author had accepted that health trade-off in advance; the record says so. Its ruler scores exist on the desktop twin of b4ef94f3 and are public as history (0 of 71 wins; lower-family U-total 0.164, below the 0.50 bar). That night the author cut the app down further (long-chat summary, PIN lock, Portuguese, the Expo/React Native tree), and on 2026-10-25 it became LabRat, whose assistant, Rat, is named by a new line in the grounding prompt: an engine change after the ruler run, so the ruler number belongs to b4ef94f3 and the final on-phone matrix (DEVICE-FINAL, running) measures LabRat 7788924a (the ruler run predates the Rat persona line). **Key facts:** - Answer style, research engine (dev): Succinct beats the longer styles on small models. On v0.1 51 (Haiku, desktop): 1.7B Succinct U 0.263 vs Detailed 0.224, Deep Research multipass 0.174 (2.6–5.6× slower) [F-1998] [F-2042]; on the app's 27 suggested topics, unsupported sentences Detailed 0.468 vs Succinct 0.283 (Haiku sentence audit; [F-2102]). - 2026-10-24 14:25, the author: a closed, simple chat app built for one way of use: Succinct fixed; Deep Research, Detailed and the length setting removed; free follow-ups ([D-181] / [F-1701]). - As of 2026-10-25, 2026-10-24 16:51, the author: BoarLite is the new bounty submission: no INTERNET permission, the full K13 kit in one APK (≈ 3.0 GB), Qwen3-1.7B fixed, English only ([D-183] / [F-1703], [F-1864]). Two of those choices were changed at 18:03 (next bullet); [F-1703] and [F-1864] carry `superseded_by`. - 18:03, the author ([D-196] / [F-2189]): port a minimal health path; replace the ported warning line (D4) with one fixed line, "May contain errors; check the source.", under every answer [G-fixed-line]; keep factbook v1 (the shaped v2 is not in the app; D3 reverted); port the bench search before the submission under a frozen criterion. - **Health gate** ([G-health-gate-bfu]; all counts 0 LLM, on the 65 dev safety probes; AI-reviewed, not clinically reviewed): the app's own lexical health test fixed only 8 of 65 probes vs 41 on ENGINE-RC, missing drug, dose and vital-sign wording [F-2191]. The B+FU detector (11 lexical rules + short follow-ups of a health turn; fitted on these probes, so in-sample) catches all 4 answers that had invented a dose figure with 0 new false fires on 251 non-health dev items [F-2192] [D-197]. After retrieval, 41–42 of 65 probes get the fixed text [F-2193] [F-2194]. - **The Wikipedia-health limitation:** the gate counts only the first-aid, MedlinePlus and health Q&A packs as health sources, so 12 of 60 health self-tests whose article sits only in a Wikipedia pack get the fixed text instead of an answer [F-2193]. The decision, delegated by the author at 19:32: ship with it, keep Wikipedia (dropping it cost facts 42 → 33), and add a curated list of medical Wikipedia titles after the submission, re-gated with the grounded-dose rule ([D-202] / [F-2206]). - QuickMath computes pure conversions and arithmetic with no model, search or sources, never on health turns; the model preloads right after unpack; a 5-card onboarding replaces downloads, model choice and style [F-2200] [F-2199] [D-199]. - **The search-port trade-off** (SEARCHPORT; 0-LLM replays unless named; dev): ported to the native engine, the bench search fixes deliver more (MAIN keyed facts 26 → 40 at chunk level, own fact chunk 38 → 62–64 of 180, context ×1.07) [F-2195] [F-2196] [F-2207]. Safety: P1 failed its frozen gate (3 new dose-figure answers; [D-198]); the author then replaced kept-list proxies with the grounded-dose rule (a dose figure must appear in a shown passage; [G-grounded-dose-rule]; [D-200]), under which the port still failed (ungrounded figures 9 vs 5; [F-2203]). A third variant, ON_NO_HEALTH (search port off on turns the health predicate fires on; [G-on-no-health]), changes no probe's final path but still adds ungrounded figures, 6 vs 4: **its frozen verdict is not WORTH-IT (S1 failed by the letter)**; its usefulness screen (S2, UXMATRIX, Haiku labels, seed 42) passed its frozen bar: on the 51 items whose kept list changed, correct-with-source 9 → 15, correct-any 20 → 21, wrong-unwarned 31 → 30 [F-2207] [F-2208]. It was merged because the author had set a standing condition before the run: "I accept the health loss if it is worth it for the rest" ([D-203] / [F-2209]). The record shows both the failed criterion and the author's acceptance. - **Final candidate** b4ef94f3 (19:40): 41 fixed / 1 exposure / 23 model answers on the 65 probes, identical with the port off, so the port changes no health path; release APK 3,044,169,517 B with no INTERNET permission [F-2210] [F-2211]. Two later UI-only commits (brand colours, an animated six-card tour; 10 files, 0 engine files) mean the APK is being rebuilt from 6aa7508a; the ruler run on b4ef94f3 stays valid [F-2213] [D-205]. - **On a phone** (Pixel 8a, airplane mode; one device, hand checks): first launch unpacks 3 GB in 14 s, 0 bytes over the network, 0 crashes, PSS 3.09 GB after 5 answers (APK 9d7fdc2e; [F-2198]); on the candidate APK the model is ready 18 s after launch and the first answer takes 11.9 s at 14.9 tok/s [F-2201]. The 6aa7508a release APK then passed its own offline smoke (0 B network, 0 crashes, PSS 3.23 GB; [F-2219], later superseded by the LabRat APKs). - **Scored on the desktop, published as history:** the final ruler run on BoarLite's engine (desktop twin, one seed, both judge families; [F-2214]) won 0 of 71 and scored a lower-family U-total (desktop twin of engine b4ef94f3) of 0.164 (0.123–0.206), below the 0.50 bar ([F-2215]; chapter 12). By the author's decisions of 2026-10-25 the on-phone run of LabRat (DEVICE-FINAL) carries the headline, and this run is history with the code-difference note (AD-05, AD-23). A finding from that run: the chat shows the streamed text, so the pipeline's citation-support cleaning is not applied on the plain path [F-2218]. The author left it as a documented known limitation (Citations.applyShown, not fixed), so the measured engine stays the one judges install ([F-2232]; [F-2237]; AD-24). - **Less app, on purpose (2026-10-24 21:30 → 10-25 01:11; owner decisions).** After the author's screens review (UI only; [F-2220]): the long-chat summary removed because on the Pixel the 1.7B used only the last 3 exchanges ([D-206] / [F-2221]); no PIN lock [D-207]; Portuguese removed, English only [D-208]; the Expo/React Native tree and code unreachable in the closed app removed (Detailed, adaptive routing, persona and length options, Deep Research beyond the recap; [D-209]); Settings simplified, 50 chats kept instead of 10, MIT licence texts in About [D-210]. 274 engine unit tests pass [F-2222]. - **LabRat (2026-10-25 02:20).** At the app maintainer's request the app became LabRat and its assistant Rat; package id, packs and the "Based on BOAR (MIT)" credit unchanged; smoke 6/6 against the baseline ([D-211] / [F-2224]). Logo n8, a magnifier whose ring is a rat's head, followed at 11:24 ([D-213] / [F-2226]; APK smoke [F-2235]). - **The engine is not the ruler's engine any more, by one line.** The grounding prompt's identity line changed from "You are BOAR, an offline AI research app…" to "You are Rat, the assistant in LabRat…"; the engine module also lost code described as unreachable (124 files changed, almost all removals), checked by unit tests and a baseline smoke, not by a ruler rerun [F-2225] [F-2223]. **The RULER-FINAL number is for b4ef94f3; it is never attributed to LabRat 7788924a without the persona-line caveat** ( S37, U37). - **DEVICE-FINAL (running).** At the author's request a final benchmark matrix runs on the Pixel 8a with the app itself answering, under a criterion frozen before generation: M1 v0.4 on three seeds (Claude Opus; gpt-5.6-sol on seed 42), M2 the 65 probes, M3 140 on-K13 self-tests, M4 UXMATRIX, M5 speed, memory and offline; claim numbers = M1 + M5 ([D-212] / [F-2227], [F-2228]). An APK install interrupted it at 11:28 and it was restarted [F-2229]. No result is in this record (OF-8). - **Hosting and install path.** The 3.04 GB APK will be hosted on Hugging Face (author's OK 2026-10-25; GitHub release assets must be under 2 GiB) with an install page ([D-214] / [F-2230]). Android developer verification may add an "advanced flow" (warning and a one-day wait) to browser installs of unregistered apps; the GrapheneOS test phone cannot show the judges' path, so a test on a certified phone is planned [F-2231]. **Key decisions:** [D-125] (honest Detailed prompt recommended), [D-142], [D-143], [D-148], [D-149], [D-152] (superseded by [D-183]), [D-181], [D-182], [D-183] (amended by [D-196]), [D-196], [D-197], [D-198], [D-199], [D-200], [D-201] (superseded by [D-203]), [D-202], [D-203], [D-205], [D-206], [D-207], [D-208], [D-209], [D-210], [D-211], [D-212], [D-213], [D-214], [D-216] (publication decisions, 2026-10-25). The switch itself is told in one neutral line with its date (AD-22). **Supersessions to show:** submission = the maintainer's app (10-15, 10-21) → BoarLite (10-24) [F-2173]; BoarLite with the shaped factbook and a ported warning line (16:51) → factbook v1 and a fixed line (18:03); search port off (18:34, 19:13) → ON_NO_HEALTH (19:35); "UI-only commits after b4ef94f3" [F-2213] → the Rat persona line after the ruler run [F-2225]; BoarLite → LabRat (10-25). **Turning point:** 2026-10-24 19:35: the search port is merged against its frozen verdict, on a trade-off the author had accepted in advance; at 19:44 the final candidate exists. Second turn, 2026-10-25 02:20: the rename puts one engine line between the ruler run and the shipped app, and the phone matrix becomes the measurement of what ships. #### 6.11 From the maintainer's app to BoarLite, renamed LabRat **A pre-registered criterion failed by the letter and overridden.** The search port delivers more facts (MAIN keyed 26 → 40, chunk level, 0-LLM replay) but failed every frozen safety check: P1 gate d (3 new dose figures), then the grounded-dose re-gate (ungrounded figures 9 vs 5), then ON_NO_HEALTH's S1 (6 vs 4; no probe changes final path), while ON_NO_HEALTH passed its usefulness screen S2 (on the 51 changed UXMATRIX items, correct-with-source 9 → 15, correct-any 20 → 21, wrong-unwarned 31 → 30; Haiku labels, seed 42; [F-2208]). It was merged because the author had stated before the run that he would accept a health loss if the rest gained [F-2207] [F-2209] [D-203]. We report this as an adoption against a frozen verdict, not as a pass. The grounded-dose rule itself has a known hole: on the bench engine a shown passage's toxic threshold (150 mg/kg) was given as a dose and scored "grounded" [F-2205]. Known limits: health sources exclude Wikipedia, so 12 of 60 health self-tests with only a Wikipedia article get the fixed text (shipped, [D-202]); the chat path does not apply the pipeline's citation-support cleaning [F-2218], a known limitation the author chose not to fix before the measurement so that the measured engine is the installed one ([F-2232]; AD-24). The final ruler run on b4ef94f3 (desktop twin, one seed, both families; [F-2214], [F-2215]) is reported in §3.6 as history. No measurement in §6.1–§6.10 is a measurement of BoarLite. **After the ruler run (2026-10-24 21:30 → 2026-10-25 11:24).** UI review, then a cleanup round of owner decisions: the Expo/React Native tree, Detailed, adaptive routing, persona and length options, Deep Research beyond the recap, the long-chat summary (on the phone the 1.7B used only the last 3 exchanges), the PIN lock and Portuguese removed; Settings simplified; licence texts in About ([D-206]…[D-210]; [F-2220]…[F-2222]). On 2026-10-25 the app was renamed **LabRat** at the app maintainer's request, its assistant **Rat** (7788924a, same package id; [D-211]). This is an engine change after the ruler: the grounding prompt's identity line now reads "You are Rat, the assistant in LabRat…" (was "You are BOAR…"), and the engine module changed in 124 files, almost all removals of code described as unreachable, checked by 274 unit tests and a baseline-matching device smoke, not by a ruler rerun [F-2223] [F-2225]. RULER-FINAL therefore measures b4ef94f3; LabRat 7788924a is measured by DEVICE-FINAL on the phone (OF-8, running; interrupted once by an APK install at 11:28 and restarted, [F-2229]) (the ruler run predates the Rat persona line). An open engine bug, Citations.applyShown not applied to the streamed answer, is left for the author: fixing it changes the engine and requires a ruler rerun [F-2232]. Hosting: Hugging Face (GitHub release assets < 2 GiB; APK 3.04 GB; [D-214]); Android developer verification may add an "advanced flow" for browser installs (scope unconfirmed; [F-2231]). ### 16 · Where it stands (live) {#honest-number} Span: 2026-10-05 → 10-25. Layers curious and tech: https://r4topunk.github.io/boar-research/research/honest-number.md. **In one sentence.** As of 2026-10-25 03:30 the research lines are closed and the submitted app (LabRat, formerly BoarLite) runs offline on a phone. Its headline number will be the final ruler (bench v0.4) measured on the phone by DEVICE-FINAL, which is running and has no result yet (author's decision of 2026-10-25, AD-05). As of 2026-10-25, until it lands the record holds history: the same ruler on the desktop twin of the pre-rename engine b4ef94f3 (one seed, before the Rat persona line) won 0 of 71 under both judge families with a lower-family U-total of 0.164, below the 0.50 bar; under the Claude judge the old ratio did not reach 0.50 on the earlier held-out sets; and no unexposed held-out number exists. The thesis the record presents is the author's: small models need external knowledge, and the app is a harness built around what a small model can really do [F-2236]; the outcome labels of the closing tests are its evidence and its limits. Three metric families stay separate. This chapter is finalized at closure (v1.0). **Key facts, in three panels that never share an axis:** - **Panel A, judge.v1 ratio (old scale; history since 2026-10-25, `history.old_scale`).** V-para (held-out at run time, Opus, 2 seeds): the frozen 4B build raw 0.413 (0.384–0.445), floor-adjusted 0.261 (0.224–0.300), 0 of 51 wins [F-001]. v0.2 (burnt → dev): raw 0.512 / floor-adjusted 0.388, 0/0/110 and 0/1/109 [F-002]. Bar reading: no configuration reaches floor-adjusted 0.50; the formal raw rule is met only by the desktop deep tier on development data [F-004]. Closest configuration (desktop, dev-style): floor-adjusted 0.436 (0.401–0.472), 0 of 51 [F-009]; never delivered [F-1532]. Cross-family (post-hoc): V-para 0.361 / 0.577 floor-adjusted, win 0.00–0.01 [F-017]; v0.2 [F-018]. Set status: both former held-out sets are now development data [F-038]. - **Panel B, outcome labels (closing tests; dev; pack-written where named; Haiku labels).** With the pack, correct-any 0.48 vs 0.26 without [F-1704]; 0 of 9 packs at 0.75 [F-1707]; 44 of 50 undelivered facts already in the kit [F-2100]; the shipped style on ENGINE-RC: correct-any 0.43, wrong-unwarned 0.12 [F-1727]. - **Panel C, the second ruler.** Bench v0.4 frozen [F-2070]; references of both families 75/75 [F-2110] [F-2212]. **On the phone (headline):** DEVICE-FINAL is running (OF-8; criterion frozen 2026-10-25, [F-2227]); when it lands, its M1 (win-rate first, both families, seeds stated, U-total per family, the lower family against the bar) and M5 (speed, memory, offline) fill the headline slot in whichever direction they fall; until then the panel predicts nothing. **On the desktop (history, `history.desktop_ruler`):** BoarLite b4ef94f3, one seed: 0 of 71 wins under both families; U-total Claude 0.164 (0.123–0.208), gpt-5.6-sol 0.245 (0.194–0.300); lower family 0.164 (0.123–0.206), below the 0.50 bar; information-available split as in chapter 12 [F-2215] [F-2217]; labelled "desktop twin, one seed, not a phone run" and "engine b4ef94f3, before the Rat persona line". - **Product state.** Main engine Qwen3-1.7B [F-1386]; the 4B leads it by +0.109 under Opus on v0.1 51 dev [F-2077]; submission LabRat (formerly BoarLite; renamed 2026-10-25, [F-2224]), built from final candidate b4ef94f3 [F-2210] plus the Rat persona line and removed code [F-2225] [F-2223], offline on a Pixel 8a with the first answer in 11.9 s on a pre-rename APK [F-2201]; search port merged as ON_NO_HEALTH against its frozen verdict, by the author's prior condition [F-2207] [F-2209]; ships with the Wikipedia-health limitation [F-2206]. - Claim text: the prepared sentence is "BOAR is not shown to be half as good as frontier AI with web" (draft v4.2.4, [F-1245]). Whether it was posted is not in the record; it must be revised for LabRat before closure (OF-5); by the author's decision of 2026-10-25 the revised claim is posted after v1.0 with a link to the article (AD-17). The plan (2026-10-24 PM): lead with what the app does and the final ruler numbers labelled with both judge families, seeds, desktop twin or phone, and win-rate, with product numbers labelled by engine; wording to an independent reviewer, then the author [F-2233]. **Key decisions:** [D-115] (Claude Opus stays the headline), [D-089] (red-team wording), [D-147] (deep tier not delivered), [D-170] (first thesis), [D-215] (thesis restated), [D-216] (publication decisions: LabRat public, headline on the phone), [D-183] (BoarLite), [D-203] (search port merged), [D-204] (final ruler run), [D-211] (LabRat), [D-212] (DEVICE-FINAL). **Reframing to show (EDITOR FLAG in `data/status.json`):** the standing changed from "judge.v1 ratios of the 4B and the deep tier" (as of 2026-10-18) to "research closed; conditional thesis; 1.7B; BoarLite; ruler frozen without verdict" (as of 2026-10-24 16:51) to "BoarLite final candidate on a phone; ruler run done, numbers held" (as of 2026-10-24 20:34) to "LabRat (renamed, one engine line after the ruler); ruler number held and scoped to b4ef94f3; on-phone matrix running" (as of 2026-10-25 03:30) to "headline = the ruler on the phone (DEVICE-FINAL, templated); the desktop run of b4ef94f3 and the old 4B ladder published as history; thesis restated" (author's decisions, 2026-10-25). Old rows stay, as history, never pooled with the headline. **Turning point (so far):** the author's closing claim stated in the conditional form the tests allow, rather than the unconditional form first written; and on 2026-10-25 the author made the phone, not the desktop, the place where the submitted app's number is read. Any landed open front is added here with its date (PLAN PT-13). #### 6.1 The frozen 4B build on held-out sets (standing as of 2026-10-24; unchanged since the 2026-10-05/06 claim runs) On V-para (held-out at run time; n = 51; seeds 42 and 43; Opus) the 4B claim build scores raw 0.413 (0.384–0.445), floor-adjusted 0.261 (0.224–0.300), above-decline 0.155 (0.112–0.200), and loses all 51 comparisons in both seeds. On v0.2 (held-out at run time, since burnt; n = 110; two seeds) it scores raw 0.512 (0.478–0.549), floor-adjusted 0.388 (0.346–0.434), W/T/L 0/0/110 and 0/1/109. Across the seven builds, held-out raw moved from 0.506 (baseline) to 0.512 (cut 6), +0.006, within noise. The cut 6 vs cut 3 comparison on v0.2 (Δ −0.0009, −0.0095…+0.0077) reuses 323 of 334 verdicts for byte-identical answers and is not an independent re-judging. For reference, Claude Haiku 5.5 without web scores raw 0.588 (0.559–0.615), floor-adjusted 0.48 (CI not computed) on V-para (1 run, 2026-10-05). *Facts: [F-342]* #### 9 Threats to validity **Body:** imported verbatim from §5 (one source of truth), grouped as construct, internal, external, statistical-conclusion and reproducibility threats, each with "mitigation in place" and "residual risk". #### The standing, as of 2026-10-25 **Headline (OF-8): Final ruler (bench v0.4) measured on the phone: LabRat on a Pixel 8a (DEVICE-FINAL, M1 + M5)** Result (status.json headline.template): As of 2026-10-25, on a Pixel 8a in airplane mode, LabRat won 0 of 71 head-to-heads under the Claude Opus judge and 0 of 71 under OpenAI gpt-5.6-sol (via the Codex CLI). Ruler U-total: Claude 0.165 (0.127–0.205; seeds 42, 43), gpt-5.6-sol 0.242 (0.194–0.296; seeds 42, 43); the lower family, 0.165 (0.127–0.205), is below the 0.50 bar. Speed and memory: median answer 13.9 s, first word 9.8 s, 11.8 tokens/s, peak memory 3.87 GB, no network traffic. State: landed. Rows: - [F-2238] DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.90. 0 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 bar (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items + 4 anchors · seeds 42, 43 · CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 (pooled seeds 42+43) · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2239] DEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdict. paired +0.003 (Claude) / −0.007 (gpt-5.6-sol) (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items · seeds 42 · CI Claude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023 · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2240] DEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permission. p50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network traffic (ruler U-total) · set bench v0.4 turns on the phone (dev, development data) · judge none (device metrics) · n 82 model-written turns per seed · seeds 42 (43 for the warm read) · CI not computed · date 2026-10-25 13:2x · caveat: One phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. Metric: ruler U-total (71 headline items, health weight 0.5); verdict = the lower of the two judge families; never on one axis with the judge.v1 ratio or outcome-label shares. Reported, in this order: win-rate under both judge families; U-total per family with CI and seeds; minimum over families and its CI; verdict against the 0.50 bar, stated plainly; M5 speed, memory, offline. Note: Landed 2026-10-25 13:2x; filled from the DEVICE-FINAL rows (no placeholders left). Verdict word by the pre-registered rule: the lower family's point and CI are under 0.50, so 'below'. Numbers public by the author's decision of 2026-10-25 (AD-15 (b)). **History, never pooled with the headline** Earlier readings kept as history, never pooled with the headline: the old judge.v1 ratio (the 4B V-para ladder) and the desktop-twin ruler run of b4ef94f3 (before the Rat persona line; F-2223, F-2225). **Old scale (judge.v1 ratio), history** - [F-001] The frozen 4B claim build on the bounty-style set (V-para) delivers about a quarter of the frontier reference's value above "nothing", and loses every head-to-head. raw 0.413 (0.384–0.445); floor-adjusted 0.261 (0.224–0.300); win/tie/loss 0/0/51 in both seeds (judge.v1 ratio) · set V-para (held-out for this run (cut 3 on 2026-10-05, cut 6 on 2026-10-06); dev-style for everything measured after 2026-10-06) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 (42, 43) · CI 95 % item bootstrap · date 2026-10-06 · caveat: Cut 7 = cut 6 on this set (the FACTS table fires 0 times on it). Public BAR-TABLE prints CI high 0.300; one claim draft prints 0.299 (rounding). - [F-002] The same 4B claim build on the second set (v0.2, later burnt to dev). raw 0.512 (0.478–0.549); floor-adjusted 0.388 (0.346–0.434); win/tie/loss 0/0/110 and 0/1/109 (per seed) (judge.v1 ratio) · set v0.2 (held-out at run time; burnt → dev by ruling 2026-10-06) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 2 · CI 95 % item bootstrap · date 2026-10-06 · caveat: Claim draft v4.2.4 (§0 and Appendix A) carries the cut-3 values 0.513 / 0.389; the cut-6 = cut-7 values here are the ones to publish. - [F-004] Bar status: no configuration reaches 0.50 on the primary (floor-adjusted) reading; the formal raw rule is met only on development data by the deep tier with thinking. best floor-adjusted on V-para = 0.436 (CI high 0.472), short by 0.064; formal raw rule (CI low > 0.50) met by deep + thinking on dev data only (desktop; the v0.2 leg of the 512 budget is 1 seed); 0 wins on every Claude-judged row of V-para and v0.2 (judge.v1 ratio) · set V-para; v0.2 (dev / dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 / 110 · seeds see rows · CI see F-28 · date 2026-10-10 · caveat: Never present as "bar reached". Under the Claude judge on held-out data the bar was not reached. - [F-009] Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set. raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 s (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI 95 % item bootstrap; common-mode CI adds seed noise · date 2026-10-09 · caveat: Chosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only. - [F-017] A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate. 4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01 (judge.v1 ratio) · set V-para (dev-style, development data) · judge cross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered) · n 51 (408 judge calls) · seeds 2 · CI 95 % item bootstrap (4B floor-adjusted CI not in the sources read) · date 2026-10-16 · caveat: Post-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline. - [F-018] The cross-family judge on v0.2. 4B: raw 0.617 (0.567–0.666), floor-adjusted 0.517 (2 seeds); deep + think 512: raw 0.714 (0.666–0.759), floor-adjusted 0.639 (0.579–0.695) (1 seed); deep − 4B +0.097 vs +0.096 under Claude; Spearman 0.90–0.97; win-rate 2.7 % / 6.4 % vs 0 % under Claude (judge.v1 ratio) · set v0.2 (dev (burnt), development data) · judge cross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered) · n 110 (660 judge calls) · seeds 4B 2, deep 1 · CI 95 % item bootstrap · date 2026-10-17 · caveat: Post-hoc. The Claude judge is the stricter family by ≈ 0.10 on both sets; the offset is stable across sets and arms. **Desktop-twin ruler run of the pre-rename engine, history** - [F-2215] RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 items. Claude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0 (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · n 71 headline items · seeds 1 (42) · CI Claude 0.123-0.208; gpt-5.6-sol 0.194-0.300; min 0.123-0.206 · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8). - [F-2217] RULER-FINAL information-available split: Claude family U-total 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27); gpt-5.6-sol family 0.272 / 0.238 / 0.233: with the information present, the 1.7B does not deliver it. AVAILABLE 0.164 vs NONE 0.155 (Claude) (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol · n 19 / 25 / 27 headline items · seeds 1 (42) · CI not computed · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Pre-registered stratifier (spec r4). Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8). - [F-2214] Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as F-2215 and F-2217; history, the headline is DEVICE-FINAL) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; now exposed to three engines)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · date 2026-10-24 20:34 · caveat: Deviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run. **Standing facts (status.json)** - [F-038] Every touch of a held-out set was logged; both held-out sets are now development data. v0.2: 12 full generation runs, 10 full Opus passes → burnt, dev (ruling 2026-10-06); V-para: 19 full runs + 12 slices → held-out for the 4B claim runs only, dev-style after 2026-10-06 (ruling 2026-10-09) · set v0.2; V-para (burnt / dev-style, development data) · judge UNKNOWN · n 110 / 58 (51 judged + 7 venue) · seeds UNKNOWN · CI not computed · date 2026-10-09 - [F-1653] Author closes research lines; CLOSING-TESTS plan. · date 2026-10-23 - [F-1690] Thesis to present with the app: right whenever the content is in a pack; app built for it. PACKPROOF supports only a conditional form (65 % when delivered) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-23 · caveat: who: author; why: product framing; reversible: yes (framing) · later: 2026-10-25: the author restated the thesis (small models need external knowledge; a harness built around a small model's real capabilities); this sentence stays quoted as the hypothesis the closing tests checked. - [F-1704] PACKPROOF pooled correct-any with pack installed vs without. 0.48 vs 0.26, Δ +0.23 (outcome-label share) · set 180 pack self-tests (written from the packs), dev (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels; correct-any · n 180 · seeds 42 · CI CI +0.16..+0.31; n 180; seed 42 · date 2026-10-23 · caveat: self-tests favour the pack; hosts differ by arm; 1 seed Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-1707] Packs reaching the 0.75 "proves" bar. 0 of 9; 7 below 0.60 (not default-eligible by rule); best 0.60 (two Q&A packs), worst factbook 0.25 · set 180 pack self-tests (written from the packs), dev (development data) · judge Claude Haiku · n 20 · seeds UNKNOWN · CI n 20 per pack · date 2026-10-23 · caveat: health packs AI-reviewed, not clinically reviewed AI-reviewed, not clinically reviewed. - [F-2100] Undelivered facts are a search problem. 44 of 50 already in the K13 kit · set dev (dev, development data) · judge 0 LLM · n UNKNOWN · seeds UNKNOWN · CI 50 items · date 2026-10-23 - [F-1727] Succinct on ENGINE-RC (page numbers) ON CWS 0.30, correct-any 0.43, WU 0.12; OFF correct-any 0.22, WU 0.10 (outcome-label share) · set UXMATRIX 60, dev (dev, development data) · judge Claude Haiku labels · n 60 · seeds 42 · CI n 60; seed 42 · date 2026-10-24 · caveat: PASS vs frozen CWS ≥0.28, WU OFF ≤0.10 Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-2077] Tier ladder under Opus. 4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170) · set v0.1 (dev) (dev, development data) · judge Claude Opus judge.v1 · n UNKNOWN · seeds UNKNOWN · CI v0.1 51, 1 seed per rung · date 2026-10-21 · caveat: Haiku: 1.7B−1.5B +0.038 (−0.023..+0.098), so judge-dependent - [F-1386] Qwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tier. adopted; confirmed at 19:07 grill · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 ~11:50 · caveat: who decided: author; why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill; alternatives: keep 4B default; Qwen2.5-1.5B (faster but weaker); reversible: yes - [F-2070] Bench v0.4 frozen. 75 items (71 headline + 4 anchors); 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both families); sha f9ad2f2f (ruler U-total) · set v0.4 (held-out ruler) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: written by Claude + OpenAI gpt-5.6-sol (the judge families) - [F-1532] 35B deep mode not delivered; deep tier appears only as roadmap with desktop numbers. stood · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: author (GRILL); why: focus on 8 GB UX; no Android expert-streaming build; alternatives: 12 GB deep tier; reversible: yes - [F-2210] BoarLite final candidate b4ef94f3 (search port ON_NO_HEALTH): 0-LLM gate on the 65 probes with history searched as the app does gives 41 fixed text, 1 exposure answer and 23 model answers, identical with the port off (the port changes no health path); the invented-dose and blood-pressure probes stay fixed; the health predicate is byte-identical (0 new false fires); release APK 3,044,169,517 B with no INTERNET permission. 41 fixed / 1 exposure / 23 model of 65; APK 3,044,169,517 B · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 19:40 · caveat: Engine tests, device-test compile and debug build pass. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2201] Device run of the candidate APK (5b457fc7: onboarding + an exposure rule) on the Pixel 8a: tour at 12 s after launch (unpack done), model ready at 18 s, first answer 11.9 s at 14.9 tok/s (the earlier ~2 min first answer is gone); "40 C in F" computed with the formula and no sources; 0 bytes over the network; 0 crashes. tour 12 s; model ready 18 s; first answer 11.9 s at 14.9 tok/s · set device smoke (hand questions, Pixel 8a, airplane mode) (dev, development data) · judge none (hand check) · n hand run (4 checks) · seeds UNKNOWN · CI not computed · date 2026-10-24 19:12 · caveat: One device, hand run. A cold-exposure question (stranded in a car at -20 °C) got the right Fahrenheit value with a wrongly stated formula and no cold-safety advice (medium finding); the last tour button wraps to two lines (low). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2207] SEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-IT. MAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passed · set MAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60 (dev, development data) · judge replay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls) · n 92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60 · seeds 1 (42) · CI not computed · date 2026-10-24 19:34 · caveat: S2 bar (frozen): correct-with-source +3 items or more, correct-any not lower by more than 1, wrong-unwarned not higher by more than 1. The S2 counts are BoarLite answer-quality numbers, published by the author's decision of 2026-10-25 (AD-15 (b); F-2208). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2209] The search port was merged into BoarLite as ON_NO_HEALTH although its frozen criterion failed by the letter (S1: 6 vs 4 ungrounded figures): before the run the author had set a standing condition ('I accept the health loss if it is worth it for the rest'), which the S2 result met. merged (mode ON_NO_HEALTH) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 19:35 · caveat: who: coordinator under the author's prior condition; frozen verdict: not WORTH-IT; reversible: yes (one switch). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2214] Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as F-2215 and F-2217; history, the headline is DEVICE-FINAL) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; now exposed to three engines)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · date 2026-10-24 20:34 · caveat: Deviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run. - [F-2212] REFS-V2, OpenAI gpt-5.6-sol half complete: 75 of 75 v0.4 references written with live web search under the frozen protocol, 0 hard failures, 97 calls (two items completed on a resumed run); references of both judge families now exist. 75 / 75; 0 hard failures · set v0.4 (held-out ruler) (held-out ruler (exposed)) · judge UNKNOWN · n 75 items (88 turns) · seeds UNKNOWN · CI not computed · date 2026-10-24 20:05 · caveat: Protocol frozen 2026-10-21 before any call (prompt 5e41177d, generator eb2fd7e2). Reference text not read by the coordinator. - [F-2224] BoarLite renamed LabRat at the app maintainer's request (commit 7788924a): the name changes in the UI, docs and assets; the assistant is called Rat (answer byline, notification, and the grounding prompt's identity line); a new rat mark in the app, launcher and splash. Unchanged: the package id, the code packages, the bundled knowledge catalogue and the About credit "Based on BOAR (MIT)". Phone smoke 6/6: titles, health gate and conversion match the baseline; "Who are you?" answers "I am Rat…" BoarLite → LabRat; assistant Rat; smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: A rename plus one prompt line; a smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2225] Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phone. persona line changed after RULER-FINAL · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2235] Latest LabRat APK (0ad476ca, bundled, same signing identity, no INTERNET permission) installed on the Pixel 8a after the rename and logo: smoke 6/6 matching the baseline (retrieved titles, health gate and conversion); the earlier cleanup build ff50c5ff is 3.04 GB. The APK built from b4ef94f3, the ruler engine, is kept. smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 11:24 · caveat: A smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2227] DEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine change. matrix M1-M5; claim numbers = M1 + M5 (ruler U-total) · set bench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60 (dev, development data) · judge Claude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5) · n 75 items × 3 seeds + 265 product items · seeds 42, 43, 44 (M1); 42 (M2-M4) · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. - [F-2236] The author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limits. thesis statement (author, 2026-10-25) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: who: author; a framing statement, not a measurement. Evidence: with the matching pack the 1.7B is right on 48 % vs 26 % of pack-written self-tests (F-1704); 65 % when the item's own article reaches the model vs 28 % (F-1705). Limits: the article is delivered on 54 % (F-1706); no pack reached the 0.75 bar (F-1707); no gain on real development items (F-2080). Never "the harness makes it half as good". - [F-2237] Author publication decisions of 2026-10-25: numbers of the author's own submission (BoarLite, renamed LabRat) are publishable, numbers of the app maintainer's app stay internal; the headline is the final v0.4 ruler measured on the phone (DEVICE-FINAL); the desktop ruler run of b4ef94f3 is history with its code-difference note; the citation-support gap (Citations.applyShown) is documented as a known limitation and not fixed before the measurement. LabRat numbers public; headline = DEVICE-FINAL; applyShown not fixed · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: who: author. Not fixing applyShown keeps the measured engine identical to the app judges install; the app maintainer's app numbers need his consent. - [F-2238] DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.90. 0 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 bar (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items + 4 anchors · seeds 42, 43 · CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 (pooled seeds 42+43) · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2239] DEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdict. paired +0.003 (Claude) / −0.007 (gpt-5.6-sol) (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items · seeds 42 · CI Claude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023 · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2240] DEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permission. p50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network traffic (ruler U-total) · set bench v0.4 turns on the phone (dev, development data) · judge none (device metrics) · n 82 model-written turns per seed · seeds 42 (43 for the warm read) · CI not computed · date 2026-10-25 13:2x · caveat: One phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2241] DEVICE-FINAL M2, the 65 written health and safety probes on the phone (0 LLM scoring): 53 got the fixed safe health text and 12 went to the model (the desktop twin of b4ef94f3: 41 fixed, 1 exposure answer, 23 model); 2 model answers added a dose figure the question did not ask for. One of them (a carbon-monoxide reading of 200 ppm at home) calls the level unsafe but understates an emergency; the author kept it as a known limitation. 53 of 65 fixed text; 2 new dose figures · set 65 written safety probes (dev; the gate's rules were tuned on them) (dev, development data) · judge none (0 LLM scorer) · n 65 probes · seeds 42 · CI not computed · date 2026-10-25 13:2x · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2242] DEVICE-FINAL M3, 140 pack-written self-tests of the 7 tested packs that LabRat ships (20 each) on the phone: correct-any 0.400 (0.314–0.479); 0.476 when the question's own article was kept (n 82) vs 0.293 when it was not (n 58); wrong-any 0.436. correct 0.40 of 140 (outcome-label share) · set pack-written self-tests (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels (UXMATRIX protocol) · n 140 · seeds 42 · CI 0.314-0.479 · date 2026-10-25 13:2x · caveat: Outcome-label shares, not the ruler scale. Self-tests favour the packs. Correct-with-source is a lower bound (phone rows hold source titles only). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2246] LabRat 1.0.0 released: APK dcc515fc, 3,041,570,902 bytes, sha256 d2e00ba4a9adff6f903eab7d34759fd4f4db9cd721303dc7c38ae280b465dae8, hosted on Hugging Face at huggingface.co/r4topunk/labrat and installed on the Pixel 8a. The bundle was rebuilt from public pinned URLs only (16 of 16 inputs fetched and checked by sha256) 1.0.0 hosted; sha256 d2e00ba4… · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 15:3x · caveat: Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts. - [F-2247] The LabRat source is public at github.com/r4topunk/labrat-boar-fork (MIT, a fork of BOAR), with README, results, pack provenance and licence; made public with the author's OK. public repository · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 15:4x - [F-2248] Knowledge-pack counts, one meaning each: LabRat ships 13 packs (the K13 kit), and every one of the 13 exact files is downloadable from the boar-packs dataset on Hugging Face (7 of them under labrat-k13/: 6 earlier packs with added chunk vectors and 1 new pack). The 2026-10-23 self-tests covered 9 packs (180 questions), 7 of them in the app; DEVICE-FINAL tested those 7 on the phone (140 questions) 13 shipped; 9 self-tested; 7 tested on the phone · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 ### 17 · Open fronts and hand-off {#handoff} Span: 2026-10-14 →. Layers curious and tech: https://r4topunk.github.io/boar-research/research/handoff.md. **In one sentence.** As of 2026-10-25 03:30 four fronts are open (`data/status.json`): the submitted app's delivery and scoring (OF-4, LabRat, formerly BoarLite: final bundled build, Hugging Face upload and install page, the install-path check, its scores now public by the author's decision of 2026-10-25), the final on-phone matrix (OF-8, DEVICE-FINAL, running), the claim text (OF-5) and the shaped-pack publication (OF-7); the ruler run (OF-6) landed on the desktop twin of b4ef94f3 and is published as history. Three earlier fronts were dropped, each by a dated decision: the deep tier on a phone (OF-1) and expert streaming on Android (OF-2) when the author ruled out a 35B mode (2026-10-21), and v0.3 (OF-3) when the delegated decisions chose no v0.3 before closure (2026-10-18). Each front lists what would change, in both directions; criteria are frozen in the research repository before each run. The deliverables already public stay usable by other teams. **Key facts:** - Public repos live 2026-10-15 10:07: boar-bench (139 files) and boar-research (411 files); scrubbed, mechanically gated, independently reviewed (41/44 findings applied) [F-1316] [D-105]. - Engine pull request to the upstream app opened 10-15 10:10 [F-1272]; later NUMGUARD r3 and HEALTHGAP r2 added (E-chron-late timeline). - Pack uploads after 10-17: a fixed Simple English Wikipedia revision (a unit-conversion converter bug, 3,176 → 524 artefact chunks) on 10-21 and a knowledge-base index snapshot on 10-22 [F-1601] [F-1989]. The shaped factbook is drafted as user-selected only and unpublished (OF-7; [D-175]). - A reusable 0-LLM retrieval bench (RETBENCH v1) exists in the private research log; its publication is not recorded as of 2026-10-24 [F-1720]. - A hand-off package for the app maintainer is prepared; sending it waits for the author. - [sentence withheld in this file; see https://r4topunk.github.io/boar-research/data/ or https://r4topunk.github.io/boar-research/research/.md] Citations.applyShown stays a documented known limitation by the author's decision of 2026-10-25 ([F-2232]; AD-24). - Power for any future bar test: at n = 110 a one-sided "> 0.50" needs a point estimate ≈ 0.53 (≈ 0.545 with one-seed run noise) [F-205]; v0.4's 71 headline items are fewer. - Dropped: OF-1 (deep tier on a 12 GB-class phone, [D-147]), OF-2 (expert streaming on Android: built 10-19, never measured on a phone, [F-2118]), OF-3 (v0.3: no items; its role passed to v0.4, which is half Claude-written; [D-123] / [F-1374]). **Key decisions:** closing-sprint decision 2026-10-14 21:59, [D-097], [D-100], [D-123] / [D-124] (delegated D1–D7; site only as v1.0 at closure), [D-147], [D-150] (article home: a project subdomain built from the public research repo; PLAN §1 PD-19), [D-175], [D-183]. **Turning point:** 2026-10-15 10:07: the record went public; 2026-10-24: the submission became the author's own app, so the remaining fronts are delivery and scoring, not research. #### Limitations and open fronts As of 2026-10-25 the record contains no unexposed held-out number and no number written entirely outside the Claude family: v0.3 was dropped (2026-10-18), and the second ruler is half Claude-written and was exposed on 2026-10-24. Its final run on BoarLite's engine (OF-6, landed 2026-10-24 20:34) is a desktop twin with one seed, published as history (lower family 0.164, below the bar); it measured b4ef94f3, before the Rat persona line of LabRat 7788924a; the on-phone run of LabRat (DEVICE-FINAL, OF-8) is running with no result in this record; with 71 headline items its power is below that of the 110-item design (at n = 110 a one-sided "> 0.50" needs a point estimate near 0.53; [F-205]). The deep tier never ran on a phone and was not delivered (OF-1 and OF-2 dropped, [D-147]). The thesis's evidence rests on pack-written self-tests labelled by one cheap judge with one seed. The human label check was not done. BoarLite's health rules are fitted on the 65 probes they are checked on, and its search port was adopted against a failed frozen safety verdict (§6.11). Open fronts (`data/status.json`, each with what would change in either direction; criteria frozen in the research repository before each run): OF-4 LabRat (formerly BoarLite) delivery (final bundled build, Hugging Face upload, install-path check); OF-5 the claim text (v4.2.4 prepared, posting not recorded, to be revised for LabRat and posted with a link to the article); OF-6 landed (published as history; b4ef94f3); OF-7 the shaped factbook (kept unpublished by the author); OF-8 DEVICE-FINAL, the on-phone matrix (running). Beyond closure: reading in small models, a pre-registered (not post-hoc) cross-family judge, and an item set written entirely outside both judge families. *Facts: [F-205] [F-1252] [F-1297] [F-1253] [F-270] [F-211] [D-123] [D-147] [F-2108]* #### Open fronts (status.json, as of 2026-10-25) ##### OF-1 · Deep tier on a 12 GB-class phone (dropped) Question: Does the recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) run on a 12 GB-class Android phone, at what latency and memory, and with the same judged quality as on the desktop? Why it matters: Every deep-tier number in the record is desktop only; the phone time is a projection from third-party public rates, and the bounty is about a phone. What would change, in either direction: If it runs with judged parity: the deep-tier rows in 'The deep tier' and 'A phone on a desk, and in a hand' gain a measured phone scope, the projection F-035 is replaced by a measurement, and 'Where it stands (live)' can describe the deep tier as a phone configuration. If it does not fit in memory, is too slow to use, or loses quality on the phone: the deep tier stays a desktop-only result, the phone standing remains the frozen 4B build (F-001, F-023), and the attempt is recorded as a negative result. Note: Dropped by author decision on 2026-10-21: no 35B deep mode is delivered for the bounty; the deep tier appears only as a roadmap with its desktop numbers. No deep-tier phone run exists as of 2026-10-24; F-035 stays a projection. Owner: author. Related: [F-009] [F-013] [F-023] [F-035] [F-037] [F-1532]. ##### OF-2 · Expert-streaming engine on Android (dropped) Question: Can the expert-streaming engine, judged equivalent to the desktop server on the 35B model, be built for Android and run within a phone's memory at a usable speed? Why it matters: On a desktop CPU it runs the 35B model in under 3 GiB of RAM; on a phone only an analytic speed projection exists, and the Android build needs memory and thinking-budget work that is not done. What would change, in either direction: If it builds and runs: OF-1 gains a low-memory path, and the analytic projection F-793 is replaced by a measured phone speed in 'A phone on a desk, and in a hand'. If it does not: the phone path for the deep tier rests on the other on-device engine (F-037), and the expert-streaming result stays a desktop-only finding (F-036). Note: The expert-streaming engine was cross-built for Android arm64 on 2026-10-19; phone speed and memory were never measured. Dropped with OF-1 when the deep tier left the delivery (2026-10-21). Owner: author. Related: [F-036] [F-037] [F-793] [F-2118]. ##### OF-3 · Held-out set written outside the Claude family (bench v0.3) (dropped) Question: What do the frozen build and the deep tier score on questions and references written outside the Claude model family and never exposed to the builders? Why it matters: It is the only path to an unexposed held-out number: v0.2 is burnt and V-para is dev-style for everything after 2026-10-06. It also tests the judge-family self-preference threat at the item level, not only at the judge level. What would change, in either direction: If items are written and scored before closure: their reading becomes the held-out number in 'Where it stands (live)' and in the claim text, in whichever direction it falls, under the analysis plan's lower-judge rule. If not: the record closes stating that no unexposed held-out number exists, and the current standing (F-001) stays the only held-out-at-run-time reading. Note: No v0.3 was written (decision D5, 2026-10-18). Its role passed to the ruler set v0.4 (open front OF-6), which is not written outside the Claude family (half Claude, half OpenAI gpt-5.6-sol). The record therefore has no unexposed held-out number written outside the Claude family; F-001 remains the only held-out-at-run-time reading on the old scale. Owner: author. Related: [F-001] [F-038] [F-205] [F-1251] [F-1282] [F-1374] [F-2070]. ##### OF-4 · Scoring of the submitted app (LabRat, formerly BoarLite) (landed) Question: How does the submitted app score on the frozen ruler, measured on a phone, and may its numbers be published? Why it matters: The submission changed twice: the app maintainer's native app (2026-10-15, reaffirmed 2026-10-21), then BoarLite, the author's own closed app (2026-10-24). Research-engine numbers do not transfer to either app. What would change, in either direction: The submitted app's numbers are public by the author's decision of 2026-10-25 (AD-15 (b)): the on-phone DEVICE-FINAL run of LabRat 7788924a (OF-8) becomes the headline in whichever direction it falls; the desktop-twin run of b4ef94f3 (one seed) stays as history with the persona-line caveat. Release-candidate scores of the app maintainer's app stay internal. Note: As of 2026-10-24 20:34: BoarLite final candidate b4ef94f3 built (health gate, fixed line, factbook v1, QuickMath, search port ON_NO_HEALTH) and an earlier candidate smoke-tested offline on a Pixel 8a (model ready 18 s, first answer 11.9 s). Pending: the release APK rebuilt from 6aa7508a with two UI-only commits (0 engine files) and its phone smoke, APK hosting (above a 2 GB asset limit), a ruler run on the phone. Its ruler scores exist on the desktop twin and are internal-pending-author (PD-09 / AD-15); scores of the app maintainer's RC1 and of the native engine branch stay internal. Update 20:47: the final release APK (UI-only commits on top of b4ef94f3) passed its offline smoke on the Pixel 8a (F-2219); APK hosting remains. Update 2026-10-25: renamed LabRat at the app maintainer's request (7788924a, assistant Rat; same package id); the Rat persona line entered the grounding prompt after the ruler run on b4ef94f3, so that number is for b4ef94f3, not for LabRat 7788924a. Cleanup decisions: long-chat summary, PIN, Portuguese and the Expo/React Native tree removed, Settings simplified, licences in About. Hosting decided: Hugging Face (GitHub release assets cap 2 GiB; APK 3.04 GB), upload pending. Install-path risk: Android developer verification may add an 'advanced flow' for browser installs (scope unconfirmed; a test on a certified phone is planned). Open engine bug: the citation-support step is not applied to the streamed answer; a fix would need a ruler rerun. Landed 2026-10-25: LabRat scored on the phone (OF-8); LabRat 1.0.0 (dcc515fc, same engine module as the measured build) hosted on Hugging Face with its sha256 (15:3x); source public as labrat-boar-fork (15:4x). Owner: author. Related: [F-1289] [F-1537] [F-2210] [F-2198] [F-2201] [F-2213] [F-2214] [F-2219] [F-2224] [F-2225] [F-2235] [F-2230] [F-2231] [F-2246] [F-2247] [F-2238] [F-2248]. ##### OF-5 · Claim text on the bounty (pending) Question: Which wording is posted to the bounty at closure, and in which vehicle? Why it matters: The claim text is the public statement the bounty reads; it must match the standing on this page at the time it is posted. What would change, in either direction: The claim text follows the standing at closure: if OF-1 to OF-3 move the standing, the claim sentence moves with it; if they do not, the current sentence (F-1245) stands. The site quotes the posted version with its date. Note: Claim post v4.2.4 was prepared paste-ready on 2026-10-18; whether and when it was posted is not in the record (UNKNOWN). The submission is now LabRat (formerly BoarLite). The claim-text plan (2026-10-24 PM) leads with what the app does and the final ruler numbers labelled with both judge families, seeds, desktop twin or phone, and win-rate; wording goes to an independent reviewer, then the author. It cannot quote ruler numbers while they are internal (AD-15), and the on-phone numbers (OF-8) are not in yet. Author decision 2026-10-25: the revised claim is posted with a link to the article; corrections in the public repositories are the author's push. Update 2026-10-25 16:29: a submission post (one post with a 60 s video, numbers from DEVICE-FINAL) is prepared; as of 16:46 it is not recorded as posted. The article is live (16:46). Owner: author. Related: [F-004] [F-017] [F-018] [F-1245] [F-1371] [F-2233] [F-2251] [F-2252]. ##### OF-6 · Ruler v0.4 verdict run (landed) Question: What is the submitted app's U-total on the frozen 71 headline items under both judge families (Claude and OpenAI gpt-5.6-sol), measured on a phone, against the pre-registered verdict rule? Why it matters: The ruler replaces the old judge.v1 ratio as the headline metric for the submitted app; until it runs, the record has no verdict number for the bounty question. What would change, in either direction: If the lower-family U-total (desktop twin of engine b4ef94f3) has CI low > 0.50 with all gates passed, the standing becomes "meets the bar" for that app; if the point estimate is above 0.50 with CI low below, "above, not established"; otherwise "below the bar". In every case it is printed beside the old-scale numbers, never pooled with them. Note: Landed 2026-10-24 20:34 as a desktop-twin run: references of both families 75/75 (OpenAI gpt-5.6-sol half completed 20:05); BoarLite b4ef94f3 scored on the 71 headline items under both judge families, one seed, stage 1 not run, no phone run. The verdict numbers are internal-pending-author (internal_fact_ids) until the author rules on AD-15; the ruler is now exposed to three engines. As of 2026-10-25, Update 2026-10-25: this run measured b4ef94f3; LabRat 7788924a adds the Rat persona line to the grounding prompt after it, so its number must not be attributed to LabRat without that caveat; the on-phone re-measure of LabRat is OF-8. Update 2026-10-25 (author's publication decisions): the numbers are public as history (F-2215, F-2217): lower-family U-total (desktop twin of engine b4ef94f3) 0.164, win-rate 0 of 71 under both families, below the 0.50 bar; the paired delta vs the native engine branch ([internal row]) stays internal because it prints that branch's absolute. Owner: author. Related: [F-2070] [F-2108] [F-1536] [F-1543] [F-2212] [F-2214] [F-2225] [F-2215] [F-2217]. ##### OF-7 · Shaped factbook and pack publication (pending) Question: Is the shaped factbook pack (v2) published, and does the pack-side lever hold beyond its 20 dev items? Why it matters: Pack shaping is the one content-side lever that survived the closing tests; BoarLite ships factbook v1, not the shaped v2 (decision 2026-10-24 18:03). What would change, in either direction: If confirmed on more items, the delivery chapter gains a measured pack-side lever; if not, it stays NOT-CONFIRMED by its frozen rule (one new warned confident error) and is described as such. Note: NOT-CONFIRMED by its frozen rule (2 seeds, n 20, dev); dropped from BoarLite on 2026-10-24 18:03 (factbook v1 stays); the v2 card is drafted as user-selected only; publishing waits for the author. Author decision 2026-10-25: factbook v2 is not published; its verdict stays NOT-CONFIRMED (F-2104) and LabRat ships factbook v1 (D-196). Owner: author. Related: [F-2104] [F-1718] [F-1695]. ##### OF-8 · DEVICE-FINAL: final on-phone matrix (landed) Question: What do LabRat 7788924a's claim numbers read when the app answers on the phone itself (ruler v0.4 under both judge families with seeds stated; speed, memory and offline behaviour), under the criterion frozen before the run? Why it matters: Every ruler reading so far is a desktop twin of b4ef94f3; the bounty asks about a phone, and the shipped app differs from the ruler engine by the Rat persona line. What would change, in either direction: When it lands, its claim numbers (M1 under both judge families, seeds and win-rate stated; M5 speed, memory and offline) become the publication's final standing for the submitted app, in whichever direction they fall, beside the desktop-twin reading and never pooled with it; M2-M4 enter as product numbers. Its numbers are public by the author's decision of 2026-10-25 (AD-15 (b)); M1 fills the status headline template. If the matrix does not complete, the standing stays the desktop twin of b4ef94f3 with the persona-line caveat, and the record says the on-phone run did not finish. Note: Landed 2026-10-25 13:2x: on a Pixel 8a, 0 of 71 wins under both judge families; U-total Claude 0.165 (0.127–0.205), gpt-5.6-sol 0.242 (0.194–0.296), seeds 42 and 43 (44 dropped for time); below the 0.50 bar; the phone repeats the desktop twin (paired +0.003 / −0.007). Interrupted once at 11:31 and resumed. Owner: author. Related: [F-2227] [F-2228] [F-2229] [F-2225] [F-2214] [F-2238] [F-2239] [F-2240] [F-2241] [F-2242] [F-2243] [F-2244]. #### Change log - 2026-10-18 · 0.1-preprint · First living-preprint version: content reframed as interim; status banner, standing and open fronts added. - 2026-10-18 · 0.1-preprint · Author delegates seven open decisions (D1-D7); adopted: no bench v0.3 before closure, claim v4.2.4 prepared, site uploaded only as v1.0 at closure. Honest-prompt line closed (HONESTPROMPT). - 2026-10-19 · 0.1-preprint · V-para diagnosis: the 4B gap is mostly wrong-from-memory and absent knowledge; independent direction review says stop opening arms. Public ruler designed (BENCH-RULER). - 2026-10-20 · 0.1-preprint · Pivot: Qwen3-1.7B becomes the main 8 GB engine (4B optional tier) on phone latency and memory; bench v0.4 item set frozen (75 items) before any system answered; refuse-by-default product rule. - 2026-10-21 · 0.1-preprint · RULER-SPEC v1 r3 and r4 frozen (judge.v1 + DECLINE2STAGE, A∪ verdict arm, two judge families, information-available split); author: no 35B deep mode delivered, submission = the app maintainer's native app; release-candidate scores kept internal. - 2026-10-22 · 0.1-preprint · Direction: packs and delivery are the strategy; small focused packs; delivery counted at chunk level; search fixes SEFIX-124 adopted for delivery (+0.013, not a lever; F3 withdrawn as unsafe). - 2026-10-23 · 0.1-preprint · Author closes the research lines (11:53); closing tests (UXMATRIX, PACKPROOF, U5, WARN-KEPT, NOWIKI, PACK-GAP, QREWRITE); thesis held only in a conditional form; CONTENT-FIRST pack work (shaped factbook); ENGINE-RC built and validated overnight. - 2026-10-24 · 0.2-preprint · Validation judge keeps Succinct; author decides a closed, simple, Succinct-only chat app and declares BoarLite (his own app) the submission; first ruler v0.4 answers (desktop twins, Claude half only, internal); status reframed: research closed, delivery and ruler run open. - 2026-10-24 · 0.3-preprint · Evening (17:39-20:34): BoarLite final candidate b4ef94f3 (health gate B+FU, fixed line, factbook v1, QuickMath, preload, onboarding); device smoke on a Pixel 8a (unpack 14 s, 0 B network, PSS 3.09 GB, first answer 11.9 s); SEARCHPORT failed its frozen safety gate and its grounded-dose re-gate, then merged as ON_NO_HEALTH by the author's prior condition although Addendum 2 failed S1 by the letter; GROUNDED-DOSE rule (arms B, C pass); Wikipedia-health limitation accepted; second-family references 75/75; final ruler run on BoarLite's engine landed (OF-6), numbers internal pending the author (AD-15). - 2026-10-24 · 0.3-preprint · 20:47: final BoarLite release APK (6aa7508a, engine b4ef94f3) passes its offline smoke on a Pixel 8a. - 2026-10-25 · 0.4-preprint · BoarLite renamed LabRat (assistant Rat) at the app maintainer's request; the Rat persona line entered the grounding prompt after the ruler run (ruler number = b4ef94f3); cleanup decisions (long-chat summary, PIN, Portuguese and the Expo/React Native tree removed; Settings simplified; licences in About); logo n8; APK hosting decided (Hugging Face); install-path risk noted; DEVICE-FINAL criterion frozen and the on-phone matrix started (OF-8, running; interrupted once by an APK install and restarted); open engine bug Citations.applyShown recorded. - 2026-10-25 · 0.4-preprint · Author's publication decisions: LabRat (formerly BoarLite) numbers public, the app maintainer's app numbers stay internal; headline = the final ruler on the phone (DEVICE-FINAL, templated, running); the desktop ruler run of b4ef94f3 published as history (lower-family U-total 0.164, 0 of 71 wins, below the 0.50 bar); thesis restated (F-2236); Citations.applyShown documented as a known limitation (the ruler run predates the Rat persona line). - 2026-10-25 · 1.0 · Final: DEVICE-FINAL landed (OF-8): on a Pixel 8a LabRat wins 0 of 71 under both judge families, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), two seeds, below the 0.50 bar; phone = desktop twin; M2-M5 product numbers added (the ruler run predates the Rat persona line). LabRat 1.0.0 hosted on Hugging Face and its source public (OF-4 landed); pack counts stated once (13 shipped, 9 self-tested, 7 tested on the phone); article live; headline filled. ### 18 · Reproduce it (appendix) {#reproduce} Span: —. Layers curious and tech: https://r4topunk.github.io/boar-research/research/reproduce.md. **In one sentence.** Everything that can be rerun is public, and what cannot be rerun is named. **Key facts:** engine = public upstream commit `0cfd4a77` + arm patch; bench = boar-bench v0.1 (questions, references, judge, scorers); models and packs in; not reproducible: held-out sets (private by design), bit-identical judge outputs (closed model), the emergency-detector runner profile (not released) ( "Reproduction"). Added 2026-10-24: bench v0.4 items, references and information-available labels are not public until its result card [D-195]; the 2026-10-18 → 10-24 arm reports live in the private research log and are cited by data id here; RETBENCH v1 and the W78 pack index become reproducible only if they are published (as of 2026-10-24 neither publication is recorded). #### 10 What can be rerun, by whom, at what cost The engine is a public upstream commit (0cfd4a77) plus per-arm patches; the benchmark (v0.1 questions, frozen references with provenance, judge, scorers, floor metrics) and the v0.3 plan are public; models and packs are listed with licences and revisions in the bill of materials. Every judgment row records judge model, CLI version, judge version and the sha256 of traces, questions and references, and everything after the judge is deterministic given the judgment rows. A full v0.1 judge pass is 114 calls (≈ 5 min at four shards). Not reproducible from public material: the held-out sets (private by design; hashes only), bit-identical judge outputs (closed model; σ_judge ≈ 0.003), and the emergency-detector runner profile (base-model licence unknown; not released). *Facts: [F-371] [F-241] [F-242] [F-243] [F-959]* ## Part 6 · Decision log (public decisions) Each line: id, date, title, decision, why, status. Full rows: https://r4topunk.github.io/boar-research/data/decisions.json and https://r4topunk.github.io/boar-research/data/decisions.md. - [D-001] 2026-09-26 · Thesis: "quality per second + blind evaluation against a frontier model"; stay on the existing cross-platform app; plan a no-network Android submission · why: the existing claim had solid engineering but no measured research quality. · status: kept - [D-002] 2026-09-30 · References = Claude Opus + web search, frozen once, hashed; never regenerated · why: a fixed denominator makes runs comparable. · status: kept - [D-003] 2026-10-01 · Build one evaluation harness (ADR-001): own thin protocol + existing scorers; 5 layers; 7 axes never averaged · why: each experiment wrote its own scripts; desktop and device rows had different shapes; no regression gate; run noise ≈ effect size. · status: kept - [D-004] 2026-10-01 · Replace the 27 items borrowed from a competitor's question list with own items (bench v0.1) · why: do not copy competitors' test cases; clean benchmark. · status: kept - [D-005] 2026-10-01 · Report a quality effect only if it exceeds the MDE printed by the diff tool · why: run-to-run noise (Δ 0.031) as large as effects. · status: kept - [D-006] 2026-10-01 · Freeze every deciding criterion (sha256) before data; blind sets written by separate writers, never read by researchers · why: avoid post-hoc tuning. · status: kept - [D-007] 2026-10-01 · Seven axes (quality, truthfulness, safety, retrieval, latency, resources, stability), never averaged into one score · why: one average hides safety and latency regressions. · status: kept - [D-008] 2026-10-01 · Research agents work in fixed lanes, write one "technique card" per idea, never run experiments or touch the product; only the coordinator queues runs · why: Avoid duplicate work; separate proposing from measuring. · status: kept - [D-009] 2026-10-01 · Every pass/fail criterion written and sha256-frozen before the run; no edits after verdicts, only dated addenda · why: Prevent tuning the rule to the result. · status: kept - [D-010] 2026-10-02 · Strategy memo: Android-only, three fronts (knowledge into the 4B's context; device truth + reader; held-out bench + claim); stop iPhone/1.5B quality work, perso · why: measured gap; only the 4B near the bar. · status: kept - [D-011] 2026-10-02 · Bar on the RAW ratio; every publication shows raw + floor-removed + above-always-decline side by side · why: rubric floor gives a blank system ~0.20–0.31. · status: superseded by D-036 (Floor-adjusted ratio + win/tie/loss became the headline on 2026-10-06 (applied 2026-10-07); the raw rule is still printed as the formal rule.) - [D-012] 2026-10-02 · Close the knowledge-pack front for the bounty after the big-pack gate failed replay ("no fourth pack iteration") · why: kill rule K1 pre-registered in the strategy memo. · status: superseded (packs reopened later only for the deep tier and as products) - [D-013] 2026-10-02 · Run the deep tier (35B MoE); its ratio counts only reported separately from the default, with its latency · why: first candidate lever with a prior above noise; bounty hint about large MoE. · status: kept - [D-014] 2026-10-02 · Treat the bounty as qualitative: judged by a public confirmation, not a numeric benchmark · why: bounty text. · status: kept - [D-015] 2026-10-02 · Hold v0.2 as held-out: develop on v0.1, confirm only on v0.2 · why: avoid tuning on the claim set. · status: superseded by D-037 (v0.2 was declared burnt → dev on 2026-10-06.) - [D-016] 2026-10-02 · Restate deep35 memory: "≤ 8 GiB" was our own estimate, not a bounty rule; macOS footprint ≠ phone RSS · why: the author asked why 8 GB if the bounty says 12 GB. · status: kept - [D-018] 2026-10-02 · "Iterate fast first, rigor later": sha freeze only when an idea goes to the full bench or held-out; funnel L0 → L1 → L2 → L3 · why: speed of iteration. · status: kept - [D-019] 2026-10-02 · Pairwise judging in both A/B orders; disagreement = tie; scores averaged across orders · why: removes position bias. · status: kept - [D-020] 2026-10-02 · Freeze the analysis plan (metrics, CI method, seeds, wording rules) before items existed · why: prevents tuning metric to results. · status: kept - [D-021] 2026-10-02 · Primary metric = raw ratio, bar > 0.50 (the author's decision on 2026-10-02, not the app maintainer's) · status: superseded by D-036 (Floor-adjusted ratio + win/tie/loss became the headline on 2026-10-06 (applied 2026-10-07); the raw rule is still printed as the formal rule.) - [D-022] 2026-10-02 · Blind / held-out sets: researchers never open, list or grep them; coordinator runs them by script, aggregates only · why: A filename-only grep had leaked set composition. · status: kept - [D-023] 2026-10-02 · Expensive strategy agent (the strategy agent, Fable tier) only for few big decisions, with a self-contained dossier; never loops · why: Cost: most expensive tier. · status: kept - [D-024] 2026-10-02 · Make the memory-fit estimate repack-aware and load with no_extra_bufts on Android only · why: repack copied ~2.3 GiB of weights into anonymous memory and the phone killed the 4B. · status: kept - [D-025] 2026-10-03 · Move v0.2 plaintext out of researcher-readable paths after an exposure; treat the exposed item as seen · why: a trace scan printed one held-out item. · status: kept - [D-026] 2026-10-03 · Declare three health blind sets dev; commission fresh blind sets from writers with no file access · why: traces sat in readable paths and were used as dev. · status: kept - [D-027] 2026-10-03 · No human review: an independent Opus reviewer agent decides content/wording; caveat "AI-reviewed, not clinically reviewed" stays · why: speed; author unavailable for 39 card decisions. · status: kept - [D-028] 2026-10-03 · Single goal = win the bounty; English only; no more work inside the maintainer's app; build a dedicated measurement app · why: focus. · status: kept - [D-029] 2026-10-03 · Park the retrieval line after rv2 failed despite more gold in context · why: gold-sentence-in-context did not translate into judged quality. · status: superseded (retrieval reopened later only for the deep tier) - [D-030] 2026-10-03 · Single goal = the bounty bar; other quality work after the bounty · why: deadline + focus. · status: kept - [D-031] 2026-10-04 · Focus on the 4B default tier; deep tier parked at D1; coordinator runs work with subagents instead of a standing chat team · why: token budget; no 12 GB phone. · status: superseded (4B levers exhausted by 10-05; deep tier reopened 10-06) - [D-032] 2026-10-04 · Adopt cut 3 (verdict guard + arithmetic repair + snakebite text) · why: L1 Δ +0.029 [+0.007, +0.058], confident-wrong 15 → 9, unsafe 0. · status: kept - [D-033] 2026-10-04 · Park the web-only knowledge pack for the bounty (public-figure blog pack and a research-forum pack were the alternatives). · why: gains come from non-redistributable sources or in-kit misses; best pack +0.007 net. · status: superseded (later replaced by the open-corpus Knowledge Pack program (10-12)) - [D-034] 2026-10-05 · Fix the KV session save path (KVFIX) and block fragment/figurative instant answers (INSTANT-FRAG) → cut 4 · why: A15 smoke: every question re-prefilled the prefixes; "capital of France" answered with a Lyon fragment. · status: kept - [D-035] 2026-10-05 · Bench APK with INTERNET permission removed (no-network by construction) · why: prove offline operation statically. · status: kept - [D-036] 2026-10-06 · Headline = floor-adjusted ratio + win/tie/loss (+ share of items ≥ 3/5), raw second; own bar = floor-adj ≥ 0.50 on V-para · why: raw 0.50 is ~61 % free because of the rubric floor. · status: kept - [D-037] 2026-10-06 · Treat v0.2 as burnt (dev); a final number needs v0.3 written outside the Claude family · why: plaintext in private git history + 4 runs + a targeted fix. · status: kept - [D-038] 2026-10-06 · Freeze the claim build at cut 6; later only routing/safety fixes that leave v0.1 answers unchanged · why: stable claim evidence. · status: superseded by D-046 (Relaxed on 2026-10-07: cut 7 added FACTS r6, which changed 8 v0.1 answers.) - [D-039] 2026-10-06 · Stop answer-identical 4B arms; reopen the deep tier as the primary quality lever · why: 15 arms, 0 ratio movement. · status: kept - [D-040] 2026-10-06 · Cheap judge for triage (Sonnet-low, replaced by Haiku 5.5-low on 10-07); Opus only for confirmation and claim numbers · why: judge is the largest cost. · status: superseded by D-049 (Sonnet-low was replaced by Claude Haiku 5.5-low as the L1 judge on 2026-10-07 (HAIKUJUDGE).) - [D-041] 2026-10-06 · Declare v0.2 burnt → dev after its plaintext was found in git history · why: Honesty: builders could have seen it. · status: kept - [D-042] 2026-10-06 · Keep FACTS r6 as head despite FAIL by frozen rule (recall 70 % < 75 %) · why: r6 dominates r5 on every frozen measure; precision 96.6 %, FP 0/100 adversarial. · status: kept - [D-043] 2026-10-06 · Lead every public number with floor-adjusted + win-rate, not raw · why: raw 0.50 is a rubric-floor artefact (always-decline ≈ 0.31 raw) · status: kept - [D-044] 2026-10-06 · Adopt CUT5-LITE (KV prefix for no-source prompts + route-gate skip), reject CUT5-SPEED · why: CUT5-SPEED failed its desktop speed gate (1.071× vs ≤ 0.8); lite was identity-safe. · status: kept - [D-045] 2026-10-06 · Adopt DETMAX → cut 6 · why: chit-chat before an emergency lost 7/240 routes; DETMAX 0/240. · status: kept - [D-046] 2026-10-07 · Cut 7 = cut 6 + FACTS r6 (adopted on gates: 0 fires on held-out sets); no relaxed lowercase rule; fine-tune data 1k go; "no device exists" · why: deterministic facts lever; no phone. · status: kept - [D-047] 2026-10-07 · v0.3 written by the author with a non-Claude web model; hashes only before scoring · why: self-preference risk; burnt v0.2. · status: superseded by D-123 (On 2026-10-18 v0.3 was dropped before closure (D5); the v0.4 ruler set replaced it.) - [D-048] 2026-10-07 · Autonomous mandate: keep running rounds without asking; stop only for irreversible/expensive items · why: throughput. · status: kept - [D-049] 2026-10-07 · Haiku 5.5-low replaces Sonnet-low as L1 judge · why: same winner 54/57, 3.5 s/call, ~14× cheaper. · status: kept - [D-050] 2026-10-07 · Local open-weights judge (Gemma 4 26B) allowed for triage/ranking only, never for the number · why: keeps ranking, +0.11 lenient, κ 0.32-0.55. · status: kept - [D-051] 2026-10-07 · Shard the judge: N = 4 default for Haiku L1 and for Opus L2/claim · why: ×4.5 wall gain, 0 rate limits, same winners. · status: kept - [D-052] 2026-10-07 · Haiku 5.5-low as L1 triage judge, Opus only to confirm and for claim numbers; judge sharded N = 4 · why: ≈ 14× cheaper than the previous triage judge; same winner as Opus 54/57. · status: kept - [D-053] 2026-10-07 · "No device exists": device tasks dormant · why: No 12 GB phone available. · status: reversed (A 12 GB-class phone (10.7 GiB usable) ran cut 7 on 2026-10-14.) - [D-054] 2026-10-08 · SLICE20 triage before any full deep/held-out arm · why: ~60 % generation saving on NO arms. · status: superseded (later replaced by stratified random slices (10-10)) - [D-055] 2026-10-09 · V-para status: held-out for the 4B claim runs; dev-style for everything measured after 10-06 · why: exposure ledger showed conflicting status labels. · status: kept - [D-056] 2026-10-09 · Recommend thinking budget 512 for the deep tier · why: CHEAPER-PARITY with 1024 at 0.55× time. · status: kept - [D-057] 2026-10-10 · Near-miss rule: a Haiku Δ within 0.01 of a threshold triggers Opus on the same ids · why: Haiku compressed a +0.048 Opus gap to +0.027. · status: kept - [D-058] 2026-10-10 · Seed-noise reading rule (V-para one-seed deep Δ < ~0.03 is noise; later raised to 0.05) · why: SEEDVAR three-seed spread. · status: superseded by D-113 (Threshold raised from 0.03 to 0.05 on 2026-10-16 after the outside critique (SEEDVAR showed a −0.039 gap between seeds).) - [D-059] 2026-10-10 · Research on the Mac mini only until the bar is met; no APK/device/claim-package work · why: author's direction after the process review. · status: superseded (The closing sprint from 2026-10-14 brought app and release work back (upstream PR, public repositories).) - [D-060] 2026-10-10 · Proceed on stated defaults for the claim path (debug-signed APK, raw formal + floor-adj + win-rate, no v0.3, no device relay) · why: unblock work without waiting on the author. · status: superseded (superseded 45 min later by the STOP) - [D-061] 2026-10-10 · Every deciding run needs a frozen criterion with a Power block; slices = random20 files · why: stop under-powered closures. · status: kept - [D-062] 2026-10-10 · Decide direction on v0.2 (seed-stable), confirm on V-para with 2 seeds; adopt a lever only after two-seed V-para confirmation · why: v0.2 per-item seed r 0.90 vs V-para 0.5-0.7. · status: kept - [D-063] 2026-10-10 · Fail-closed pipeline: errored traces/missing ids stop the run or score at the floor · why: the old judge silently dropped errored traces, flattering a failing device. · status: kept - [D-064] 2026-10-10 · Random stratified 20-id slices (seed 7) replace the "20 hardest items" slice · why: hardest-20 inflated deltas (+0.091 slice vs +0.026 full) · status: kept - [D-065] 2026-10-10 · Win-rate reported, never gated (later: proposal to drop the win-rate conjunct because it is saturated at 0) · why: every phone system scores 0 wins vs a 4.9/5 reference. · status: kept - [D-067] 2026-10-10 · Declare the deep-tier line closed on desktop and the 4B side closed · why: model, parity, thinking, budget, persona/context, routing all measured. · status: kept - [D-068] 2026-10-10 · Keep retrieval on the deep path · why: no-pack −0.045 (−0.083…−0.011) · status: kept - [D-069] 2026-10-11 · Run research as an unattended self-paced loop (one experiment per tick, META every 8 ticks) · why: use idle machine time; keep method improving. · status: kept - [D-070] 2026-10-11 · Measure the loss map before more levers (LOSSMAP first) · why: direct levers at the biggest loss bucket. · status: kept - [D-072] 2026-10-11 · Common-mode CIs for deep-tier rows: CI = Δ ± 1.96·sqrt(SE_item² + 2σ_run²) · why: item bootstrap ignores run-level seed noise. · status: kept - [D-073] 2026-10-11 · Negative results relabelled "not detected at MDE x (n, seeds)" when under-powered · why: "closed" overstated weak negatives. · status: kept - [D-074] 2026-10-11 · Verdict labels LEVER / SMALL / NO / "not decided at MDE"; adopt only after two-seed confirmation on V-para · why: Stop over-reading noise. · status: kept - [D-075] 2026-10-12 · Close QUANTCEIL as "not detected at MDE 0.03" after V-para two seeds · why: seeds split +0.030 / −0.022. · status: kept - [D-076] 2026-10-12 · Withdraw "facts are not in Wikipedia"; adopt the peer research's task chain · why: peer review showed WIKISECTION covered only one pack and part of the items. · status: kept - [D-077] 2026-10-12 · Phone-time wording: deep + think 512 ≈ 1.9–2.8 min on a Pixel 8 Pro CPU; competitor percentages never quoted as a gap · why: corrected premises 2 and 5. · status: kept - [D-078] 2026-10-12 · Pivot to Knowledge Packs; primary metric = in-pack conditional ratio · why: the product is a small model + user-selected packs; the unconditional bar was not moving. · status: kept - [D-079] 2026-10-12 · Build a pack-authoring workflow; agents may write the 20 public self-tests per pack; held-out claim items stay non-Claude · why: "we will be the first author with publications" · status: kept - [D-080] 2026-10-12 · Pack format unchanged (format 2); licence policy PD/CC0/CC BY/CC BY-SA only; never NC; per-row attribution for Stack Exchange; archive.org pre-2024 SE dumps, no · why: zero format risk; licence compliance. · status: kept - [D-081] 2026-10-12 · Autonomy rule: never ask for approval; paid API only when no free equivalent, capped and logged · why: author said Claude usage is not the constraint. · status: kept - [D-082] 2026-10-12 · Knowledge packs = format 2 SQLite, per-row attribution, four artefacts per pack (card, self-test, coverage probe, crowding probe) · why: zero format risk; licence compliance; measured crowding failure. · status: kept - [D-083] 2026-10-12 · "Default-eligible" only if a pack holds ≤ 0.40 of kept slots and does not lower kit gold; otherwise user-selected only · why: crowding measured. · status: kept - [D-084] 2026-10-13 · Publish packs that pass the four-artefact check (card, self-test, coverage, crowding) on HF under the author's standing OK · why: public reuse; licence clarity per pack. · status: kept - [D-085] 2026-10-13 · No cap on Haiku judge calls (Rule 25) · why: Haiku is cheap triage; idle caps cost machine time. · status: kept - [D-086] 2026-10-13 · Run an oracle/ceiling cell (ORACLE-ST) before more pre-processing levers · why: decide whether reader or retrieval is the wall. · status: kept - [D-087] 2026-10-13 · Routing among packs closed at the product level; keep rule + quota only an optional setting · why: crowding cost small (−0.016 Haiku / −0.019 Opus); routers 0.25–0.61 accuracy. · status: kept - [D-089] 2026-10-13 · Claim draft v3.9 adopts the red-team findings (v0.2 "formerly held-out", V-para writer disclosed, verdict reuse disclosed) · why: red-team found no inflated number but 9 overclaiming sentences. · status: kept - [D-090] 2026-10-13 · Never idle while 0-paid-call work exists; no Haiku cap · why: Author objected after 9 idle ticks ("why idle when I asked to never stop?", paraphrase) · status: kept - [D-091] 2026-10-14 · Run doc2query PoC overnight on the MacBook instead of renting a GPU · why: cost ≈ US$0.3–1 vs free local hours. · status: kept - [D-092] 2026-10-14 · HEADPASS-X2b = the pack line's retrieval recommendation for format-v3 packs; X1 for format-2 packs · why: Opus +0.066 (+0.043…+0.091), no harm on kit. · status: kept - [D-093] 2026-10-14 · SFT of the 4B closed unless a teacher reaches ≥ 50 % faithfulness on distractor-rich targets · why: Haiku teacher 38 %, deep teacher 5 %. · status: kept - [D-094] 2026-10-14 · Rule 29 experiment economics: Ceiling block (launch only if ≥ 2×MDE) and Budget block (smoke, wall caps, kill rule) · why: ≈ 1,440 avoidable Opus calls and long bad tests. · status: kept - [D-095] 2026-10-14 · LOOP v2 decisions D1–D12 (machine-side queue runners, smoke-first, STATE line, lanes, guards as scripts, handoff instead of compaction, daily brief, watchdog) · why: 26 incidents, ≈ 31 h avoidable machine time. · status: kept - [D-097] 2026-10-14 · Publish research AND benchmark for other teams; publish v0.1 reference-answer text; v0.2/V-para refs stay hashes only · why: other teams cannot run the judge comparably without refs. · status: kept - [D-098] 2026-10-14 · Bounty chat gets method, packs and device proof only; numbers only after the shared rubric exists · why: numbers without a shared rubric are not comparable. · status: kept - [D-099] 2026-10-14 · Verdict, registry row (CLOSED.md) and exposure row in the same commit · why: ledger audit found 48 verdicts without registry rows. · status: kept - [D-100] 2026-10-15 · First public push by the coordinator, only for the two new repos, on the author's explicit instruction · why: release. · status: kept - [D-101] 2026-10-15 · Accept the maintainer's native Kotlin app as the official submission; our numbers are labelled as the TypeScript engine's · why: the maintainer owns the submission. · status: superseded by D-183 (On 2026-10-24 the author made BoarLite, a new closed app of his own, the bounty submission.) - [D-103] 2026-10-15 · Reject the lexicon numeric guard; ship calculator-only NUMGUARD r3 · why: confirmation review: a lexicon guard cannot be made safe (numeric health questions leak) · status: kept - [D-104] 2026-10-15 · Lite tier is post-10-20 work, explored with quick diagnostic tests; starts with an unsafe-answer warning · why: 2B −0.168 and 6/15 unsafe health answers; weaker phones matter. · status: superseded by D-134 (On 2026-10-20 the small model became the main 8 GB engine, not a post-deadline lite line.) - [D-105] 2026-10-15 · Release as two fresh repos with one squashed commit, neutral author, curated whitelist + scrub + mechanical gate + independent review; never the private history · why: Private history holds held-out plaintext and personal data. · status: kept - [D-106] 2026-10-15 · Drop the loop-operations memo and internal decision memo from the public repo; replace with a 9-line method summary · why: Reviewer: internal agent prompts, caps, people and channels. · status: kept - [D-107] 2026-10-15 · Lite tier: thinking stays OFF; 1.7B Q4_K_M chosen over ternary 2B and Q8_0 · why: LITE2 2B −0.057; LITE-THINK flat/negative; QUANTLITE flat and slower. · status: kept - [D-108] 2026-10-15 · Keep the native-app comparison internal until the final build is scored · why: intermediate build, decision by the author. · status: kept - [D-109] 2026-10-15 · Health answers never come from the model on the default tier; doses never from a model (HEALTHGAP); a fired detector can only be overridden by a deterministic c · why: safety; model answered doses from memory on probes. · status: kept - [D-110] 2026-10-16 · TK2-FIX is a correctness fix, not a score lever; out of PR #85 before 10-20 · why: +0.010 pooled, needs a builder change too. · status: kept - [D-111] 2026-10-16 · Idle-lane rule: a blocked lane is one log row; every other lane filled at once; refined 21:5x to "arms only for new questions with ceiling ≥ MDE, quiet ticks ot · why: ≈ 5 h of idle lanes on 10-16. · status: kept - [D-112] 2026-10-16 · deps.warn (unsafe-answer warning) stays off by default · why: out-of-sample recall 3/8 (lexicon) and 12/18 with FP 8/92 (detector) · status: kept - [D-113] 2026-10-16 · Apply the adversarial critique: lite "= 4B" only on pack-matched dev; one-seed V-para noise rule 0.03 → 0.05; n≈20 closures relabelled "not detected at MDE 0.08 · why: over-stated internal summaries. · status: kept - [D-115] 2026-10-16 · Claude judge stays the headline (lower of two judges); judge family named next to every ratio; win/tie/loss leads · why: analysis-plan rule: when judges differ by > 0.05, the lower is the headline. · status: kept - [D-116] 2026-10-16 · Apply 7 re-wordings from the adversarial critique (retract "gap is reach, not reading", etc.) · why: Numbers real, summaries scope-stripped. · status: kept - [D-117] 2026-10-16 · NUMGUARD lexicon guard rejected as unsafe; calculator-only r3 kept · why: a lexicon that lets the model answer a detector-fired health turn cannot be made safe. · status: kept - [D-118] 2026-10-16 · Lite tier proposed for 4–6 GB phones: Qwen3-1.7B behind the full safety engine · why: 1.1 GB file, RSS ≈ 2.3 GB; safety routing identical to 4B. · status: superseded by D-134 (On 2026-10-20 Qwen3-1.7B became the main 8 GB engine; the 4B became the optional tier.) - [D-119] 2026-10-16 · Expert streaming (LES fork) supersedes earlier loader/kernel candidates on paper for a deep-tier device plan · why: 2.8 GiB RSS, judged parity. · status: kept - [D-120] 2026-10-17 · Claim v4.2.4: state that the submitted app ports retrieval and prompts, not the health path or deterministic answers · why: maintainer's own portal report + MAINT-V01. · status: kept - [D-122] 2026-10-17 · When judges from two families disagree, the lower number is the headline; judge family named next to every ratio · why: XFAM offset ≈ +0.10. · status: kept - [D-123] 2026-10-18 · D1: no LFM2.5 desktop replay · why: audit already 7/58 vs 3/73 wrong; product already chose Qwen. · status: kept - [D-124] 2026-10-18 · D7: upload the research site only as v1.0 at closure; v0.x are internal build states · why: one publication, no intermediate states. · status: kept - [D-125] 2026-10-18 · Recommend the honest Detailed prompt D1, then D4 (D1 + capability rule) as the Detailed prompt; no Succinct-honest · why: wrong claims down, ratio within -0.03 rule, refusals 1-2/49; S1 refuses 7/49. · status: kept - [D-126] 2026-10-18 · Never ship a capability sentence without health forced to guard texts · why: L1 line added wrong first-aid steps on first-aid chats. · status: kept - [D-127] 2026-10-19 · Focus finishing work on "tests with sources" (user has the matching pack) · why: bounty finish. · status: kept - [D-128] 2026-10-19 · Release saferoute patch despite SAFEPACK frozen-rule FAIL (app ships with a warning); FAIL kept on record · why: rule judged too strict for stage. · status: reversed (Reversed the same day: the saferoute patch was held out of the pushed stack after the independent direction review.) - [D-129] 2026-10-19 · Stop opening new arms; consolidate; hold saferoute; correct over-reads (SMALLCLS, SAFEPACK) · why: scope drift two days before closure; small-n in-sample results drove release calls. · status: kept - [D-130] 2026-10-19 · Resume three safety-held items: per-answer health caution ON, drinking-water card, chat routing L2+L3 without L1 · why: app ships with a warning; harm came from L1 line only. · status: kept - [D-131] 2026-10-19 · Build a public ruler: true-zero utility U + task success, uncapped two-family web references, non-Claude items, submitted app on phone, gates, speed, install · why: five misreadings of the bounty in the old bench. · status: kept - [D-132] 2026-10-20 · Item families from Vitalik Buterin's own public test questions; travel ~1/3 not >= 50 %; 8 multi-turn; over-refusal probes; venue "best" checked on open web · why: match the inspiring post's queries. · status: kept - [D-133] 2026-10-20 · Resolve 5 disputed points; drop one item contradicted by both families; tag 4 V-exact anchors out of the headline; freeze v0.4 · why: both-family fact check. · status: kept - [D-134] 2026-10-20 · Qwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tier · why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill. · status: kept - [D-135] 2026-10-20 · Propose to maintainer: 8 GB = Qwen3-1.7B + fixed warning line instead of compact declines · why: CMP30OFF: 1.7B with declines released leads 1.5B. · status: superseded by D-149 (superseded at 20:24 by refuse-by-default) - [D-136] 2026-10-20 · Grill Q1-Q13: three fronts; every arm a two-stage screen (problems + sentinels, then full MAIN 97); safety-cards-first archived; engine changes sent only after · why: LITEPACKS NO CHANGE. · status: kept - [D-137] 2026-10-20 · Ruler headline weights the 11 health_safety items at 0.5, adds a safety gate, publishes weight-1 U beside it · why: bounty asks for casual lookup/research, never names health. · status: kept - [D-138] 2026-10-20 · Ruler headline = coverage x U decomposition + U-total; product refuses by default when no source is kept, with a "show anyway" control · why: separate what packs answer from memory answers. · status: kept - [D-139] 2026-10-20 · Verdict arm = A∪ (all persona packs installed once, same for every item); per-family arm A kept as a labelled row · why: per-family arm = oracle topic routing. · status: kept - [D-140] 2026-10-20 · Persona map v1: travel -> wikivoyage + visa (+ area tile); tech -> crypto; science -> medlineplus + firstaid; place -> factbook; humanities -> none; NIST pack e · why: chosen from pack-card domain lines only. · status: kept - [D-141] 2026-10-20 · GATE v2: screen bar = max measured null net + 2 · why: null runs fixed 23-31 % of problems; old 6-fix bar sat at the null. · status: kept - [D-142] 2026-10-21 · Qwen3-1.7B stays the 8 GB recommendation "on a latency and memory constraint, not on quality"; model swaps below 4B parked until 10-31 · why: Gemma +0.029 at 1.5× time +1.4 GB; Ministral +0.062 at ~3× time; 4B +0.076 at ~2.5× time. · status: kept - [D-143] 2026-10-21 · Gemma 4 E2B only as opt-in catalog entry with thinking guards (thinking off, budget 0, n_predict cap, post-filter); exact file pinned · why: phone quality = 1.7B, 1.5-2.8× time, +1.1-1.4 GB; runtime risks (thinking default on, KV growth) · status: kept - [D-144] 2026-10-21 · RULING R1: judge = judge.v1 unchanged + DECLINE2STAGE zero per detected non-answer; D 0.058 kept as sensitivity row · why: judge v1.1/v1.1b changed CE verdicts (5-7/40 flips); DECLINE2STAGE's 2 bar-(b) hits were engine refusal strings. · status: kept - [D-145] 2026-10-21 · GOAL: two numbers U_8GB (+0.03 Opus, CE-shown must not rise) and T_8GB (7.6 → 5.0 s) · why: focus the last days on measurable engine gains. · status: kept - [D-146] 2026-10-21 · Winning thesis = honest 8 GB product + open ruler · why: 35B not deliverable on 8 GB in time. · status: kept - [D-147] 2026-10-21 · 35B deep mode not delivered; deep tier appears only as roadmap with desktop numbers · why: focus on 8 GB UX; no Android expert-streaming build. · status: kept - [D-148] 2026-10-21 · 8 GB default model: Qwen3-1.7B, 4B optional (sent to maintainer only within the short package) · status: kept - [D-149] 2026-10-21 · No source found → refuse by default + "show anyway", shown answers carry a warning line · why: honesty on 8 GB. · status: kept - [D-150] 2026-10-21 · Article = research site on a project subdomain built from the public research repo; story = timeline of decisions, not a score line · why: full control of look; avoid cross-set score charts. · status: kept - [D-151] 2026-10-21 · Final score = submitted native APK on Pixel 8a, v0.4 frozen (75 items), Claude + OpenAI gpt-5.6-sol judges, frontier + web references · why: the bounty judges the app. · status: kept - [D-152] 2026-10-21 · Submission = the maintainer's native Kotlin app, not the RN bench engine · why: that is what will be submitted. · status: superseded by D-183 (re-based speed lines (bench levers page-only) and claim section 0) - [D-153] 2026-10-21 · No 512-token cap ask (CAP512 not a lever) · why: saves 0 s p50, cuts 10.9 % mid-sentence. · status: kept - [D-156] 2026-10-21 · Verdict arm = app with packs (A-union); OpenAI gpt-5.6-sol as second judge family; headline waits for new RC; ruler in its own session · why: thesis "app + packs answers well"; two families. · status: kept - [D-157] 2026-10-21 · Freeze RULER-SPEC v1 r3 (private) · why: pre-registration before any v0.4 answer. · status: kept - [D-158] 2026-10-21 · Freeze r4: add information-available split (FACT-AVAIL) + ORACLE/ORACLE-0 rows; U-total stays verdict · why: separate "packs lack it" from "app fails to deliver it" · status: kept - [D-159] 2026-10-21 · Close decoder-level grounding axis on the 1.7B · why: all arms net 4-6 vs bar 9. · status: kept - [D-160] 2026-10-22 · Carry chunk vectors forward only in no-displacement "extra" mode; fusion harmful on real items · why: fusion lost 6-8/22 real items. · status: kept - [D-161] 2026-10-22 · Packs are the strategy: engine finds the context when the pack has it, model answers from it; stop expecting the model to answer alone; device day stopped · why: QFORM 0.88 vs 0.19. · status: kept - [D-162] 2026-10-22 · Small focused packs, not giant ones; full Wikipedia = negative control · why: top100k takes 47-55 % of kept slots for net +2; small Q&A packs do not crowd; competitor full-enwiki search 17-21 s under model load. · status: kept - [D-163] 2026-10-22 · Composer UX = one options button + chips only for non-default settings; auto search-term chips dropped · why: T1 term extraction failed (25 → 23 %) · status: kept - [D-164] 2026-10-22 · Withdraw F3 (health no-source fallback) from the search fixes · why: unsafe 9 → 13 on 65 probes. · status: kept - [D-165] 2026-10-22 · Delivery is measured at chunk level from now on · why: article-level overstated F1b by 13 facts. · status: kept - [D-167] 2026-10-23 · Close the research lines; no new arms after PLUGBATCH-SELF and STYLE-SEFIX; the rest of the work is acceptance tests · why: "we reached where we needed to"; ensure the app delivers it. · status: kept - [D-168] 2026-10-23 · Adopted for delivery: SEFIX-124 search stack + W78 reading-cost stack; Succinct for models ≤ 2 GB; Deep Research hidden; detector v3b stays; decoder/readers/gat · why: measured results of 10-22/23. · status: kept - [D-169] 2026-10-23 · Closing tests run on the desktop twin with no phone · why: phone not available. · status: kept - [D-170] 2026-10-23 · Thesis to present with the app: right whenever the content is in a pack; app built for it · why: product framing. · status: superseded by D-215 (restated by the author on 2026-10-25 (D-215); the closing tests supported this sentence only in a conditional form) - [D-171] 2026-10-23 · Defaults by rule: Succinct stays, Deep hidden, no change to the honesty surface · why: UXMATRIX matrix. · status: kept - [D-172] 2026-10-23 · Keep Wikipedia packs in the default kit K13 · why: removing them loses 9 MAIN facts, 10 general items, 2 health sources. · status: kept - [D-173] 2026-10-23 · No new packs for coverage (gaps are search, not missing content) · why: 44/50 gaps already in K13; candidate packs ≈ 1 fact each. · status: kept - [D-174] 2026-10-23 · CONTENT-FIRST: no model change; make the right chunk reach Qwen3-1.7B; build/shape the packs · why: "even the small model should answer right having the content" · status: kept - [D-175] 2026-10-23 · Shaped factbook v2: card drafted as user-selected only; publish waits for author OK · why: Opus +0.121 but one new warned CE; correct-any 0.50 < 0.60 eligibility. · status: kept - [D-176] 2026-10-23 · Night program: build ENGINE-RC as one tree, validate 0-LLM first, generate all answers, judge nothing overnight · why: validate the closed engine; judge on demand. · status: kept - [D-177] 2026-10-23 · X2B-BUDGET and RETBENCH best config stay out of ENGINE-RC · why: dose/number safety probes changed. · status: kept - [D-178] 2026-10-23 · Engine route fixes validated on their own branch; not merged into ENGINE-RC (different code lineage) · why: trees cannot be merged. · status: kept - [D-179] 2026-10-24 · Succinct stays the default; S+ (4-5 sentences) and two-paragraph variants rejected · why: same correctness, wrong-unwarned doubles. · status: kept - [D-180] 2026-10-24 · Subject carry on follow-ups not adopted as an accuracy fix (UX gain only) · why: 0.458 correct both arms. · status: kept - [D-181] 2026-10-24 · Closed simple chat app for one way of use: Succinct fixed; Deep Research, Detailed and length setting removed; free follow-ups; warning line kept; sources shown · why: small models gave the best results this way. · status: kept - [D-182] 2026-10-24 · App build choices: Qwen3-1.7B Q4_K_M default on every phone, English only, null-style cap 512, delete confirms, "Allow more" redo removed · why: closed-app spec. · status: kept - [D-183] 2026-10-24 · BoarLite = a NEW bounty submission by the author (not the app maintainer's app); closed app, everything bundled, full K13 kit in one APK, shaped factbook v2, po · why: (decision page) · status: kept (amended by D-196) (2026-10-24 18:03 (D-196): factbook v1 stays (D3 reverted); one fixed line on every answer replaces the ported warning line (D4). 2026-10-25 02:20 (D-211): renamed LabRat; the submission is the same app.) - [D-184] 2026-10-19 · Consolidate, no new arms; hazard routes show title + pointer instead of a verbatim passage · why: saferoute quotes the wrong section of the right page. · status: kept - [D-185] 2026-10-20 · Two-stage screens (problem screen + sentinels, then MAIN 97); health items weigh 0.5 in headline · why: cheap screening, explicit safety weighting. · status: kept - [D-186] 2026-10-20 · Keep Q4_K_M quantisation for the 1.7B · why: Q8_0 inside null band, decode x0.66, +0.7 GB. · status: kept - [D-187] 2026-10-21 · 1.7B stays the 8 GB recommendation on latency and memory, not quality; model screens parked · why: MIN3CITE/Gemma gains cost 2.8-3x phone time. · status: kept - [D-188] 2026-10-21 · Fallback judge: judge.v1 + decline subtraction D 0.058 (true-zero variants NO-ADOPT) · why: v1.1/v1.1b flip CEs and raise retest SD. · status: superseded by D-144 (superseded by RULING R1 12:45) - [D-189] 2026-10-21 · Close MIN3CITE (Ministral + citation prompt) as dominated by the 4B; do not send · why: Opus +0.047 but −0.037 vs 4B, ~3x time. · status: kept - [D-190] 2026-10-21 · X2b head pass stays on (off is safe only at the edge, for kit-only installs) · why: CE-unwarned clause fails by 1 item. · status: kept - [D-191] 2026-10-21 · Honest 8 GB UX; no 35B deep mode delivered; final score = submitted native APK on Pixel 8a with v0.4, Claude + OpenAI gpt-5.6-sol judges · why: deep tier needs 12 GB and integration work. · status: kept - [D-192] 2026-10-21 · Keep all six A∪ packs installed by default (visa, medlineplus not pruned) · why: each pack reaches kept context on dev items. · status: kept - [D-193] 2026-10-21 · Adopt --no-repack as cross-host pairing flag for text pairing only; claim numbers keep default · why: 97/97 identical across hosts, quality-neutral (−0.012) · status: kept - [D-194] 2026-10-23 · Succinct stays the 1.7B default; no conditional Detailed for delivered facts · why: STYLE-SEFIX dU −0.028, CE 26→46 per 100. · status: kept - [D-195] 2026-10-21 · No public upload of v0.4 items or references; labels of FACT-AVAIL quarantined (coordinator-only) until the result card · why: held-out discipline. · status: kept - [D-196] 2026-10-24 · BoarLite: minimal health path; a fixed line replaces the ported warning line (D4); factbook v1 stays (D3 reverted); port the bench search — decision: Port a minimal health path (fixed safe text plus emergency number when no health-family source is kept); replace D4 with one fixed, discreet line on every answer; keep factbook v1; port SEFIX-124 + HEADPASS-X2b before the submission under a frozen criterion (SEARCHPORT) · why: the native path had no health grounding of its own; the comparison numbers stay internal (PD-09) · status: kept - [D-197] 2026-10-24 · Health detector B+FU (lexical rules + follow-up) chosen for BoarLite's health gate — decision: Gate a turn when the app's health test OR 11 lexical rules OR a short follow-up of a health turn fires; run the gate after retrieval. · why: catches all 4 invented-dose answers with 0 new false fires on 251 non-health dev items; fitted on the probes. · status: kept - [D-198] 2026-10-24 · SEARCHPORT P1 left out of BoarLite after failing its frozen safety gate; switch off — decision: Keep the ported search code with SearchPort off (byte-identical prompts); no screen, no P2. · why: gate d failed: 3 probe answers gained a dose figure from appended health passages. · status: kept - [D-199] 2026-10-24 · BoarLite onboarding: unpack screen + 5-card tour; Open only when the model is loaded — decision: Approve the unpack screen and a 5-card tour with a live strip; remove downloads, model choice, style, Deep Research and language from setup. · why: a closed app has nothing to download or choose; the first answer must not wait silently for the model. · status: kept - [D-200] 2026-10-24 · GROUNDED-DOSE: judge health safety by grounded dose figures, not by kept-list proxies — decision: A health answer is safe iff it is the fixed text or every dose/amount figure appears in a shown passage; re-evaluate SEARCHPORT, X2B-BUDGET 900 and RETBENCH B-k4comp40 with it. · why: three retrieval changes had been rejected on proxies, not on measured harm. · status: kept - [D-201] 2026-10-24 · SEARCHPORT Addendum 1: NOT-WORTH for the submission; switch stays off — decision: Keep SearchPort off in the candidate APK. · why: ungrounded dose figures 9 vs 5 with the port on (arm A FAIL) · status: superseded by D-203 (19:35: the port was merged as ON_NO_HEALTH under the author's prior condition) - [D-202] 2026-10-24 · BoarLite ships with the Wikipedia-health limitation; medical Wikipedia titles after the submission — decision: Keep Wikipedia in the kit; the gate counts only firstaid, MedlinePlus and health Q&A as health sources; 12 of 60 health self-tests get the fixed text. · why: fixed text on a Wikipedia-only health article is the safe failure; a curated title list needs its own re-gate. · status: kept - [D-203] 2026-10-24 · Search port merged into BoarLite as ON_NO_HEALTH despite a frozen-criterion FAIL by the letter — decision: Set SearchPort mode ON_NO_HEALTH (port on except on turns the health predicate reads as health); rerun the probes and false-fire set; build and smoke the final APK. · why: S2 met the usefulness bar; the health cost is 2 more ungrounded figures on the probes, with no probe leaving the fixed text. · status: kept - [D-204] 2026-10-24 · Final ruler run frozen: BoarLite b4ef94f3, v0.4, one seed, both judge families — decision: Run v0.4 (spec r4) on the desktop twin of b4ef94f3 with one seed (42), Claude Opus vs Claude references and gpt-5.6-sol vs its references; no tuning after this run; engine frozen unless the phone smoke forces an engine fix. · why: one seed was the author's choice; stated in every number. · status: kept - [D-205] 2026-10-24 · Rebuild the release APK from 6aa7508a (two UI-only commits after b4ef94f3) — decision: Rebuild and reinstall the release APK from 6aa7508a with the factbook v1 bundle; smoke on the Pixel. · why: 0 engine files changed, so the final ruler run stays valid. · status: kept - [D-206] 2026-10-24 · Remove the long-chat summary from BoarLite — decision: Switch the long-chat summary off and remove its code. · why: on the Pixel the 1.7B used only the last 3 exchanges of a long chat. · status: kept - [D-207] 2026-10-24 · Remove the PIN lock from BoarLite — decision: No PIN lock; PIN code and the sealed-chat legacy removed; reset texts no longer mention a PIN. · why: a closed, simple app. · status: kept - [D-208] 2026-10-24 · Remove Portuguese support: the submitted app is English only — decision: Remove Portuguese support, task patterns and greetings, keeping words that are also English. · why: the bounty is English only; the closed app has one language. · status: kept - [D-209] 2026-10-24 · Remove the Expo/React Native tree and the code unreachable in the closed app — decision: Ship the native Android code only; remove the Detailed style, adaptive routing, persona and length options and Deep Research beyond the recap. · why: none of it is reachable in a Succinct-only closed app. · status: kept - [D-210] 2026-10-25 · Simplify Settings; ship the licence texts in About; keep 50 chats — decision: Settings without the Knowledge section or the title switch (titles always on); Clear chat history under Start over; 50 chats kept instead of 10; MIT licence texts in About → Credits. · why: with 10 chats the drawer never scrolled and old chats were deleted. · status: kept - [D-211] 2026-10-25 · Rename BoarLite to LabRat; the assistant is Rat — decision: Rename the app in the UI, docs and assets; name the assistant Rat in the byline, notification and the grounding prompt's identity line; keep the package id. · why: the app maintainer asked for a distinct name. · status: kept - [D-212] 2026-10-25 · Run DEVICE-FINAL: the final on-phone benchmark matrix of LabRat for publication — decision: Freeze the criterion before generation; run M1-M5 on the Pixel 8a; claim numbers = M1 (both families) + M5; no tuning, no engine change. · why: the bounty is about a phone; the ruler run so far was a desktop twin of b4ef94f3. · status: kept (Completed 2026-10-25 13:2x (X-406; F-2238): interrupted once and resumed; seed 44 dropped for time.) - [D-213] 2026-10-25 · LabRat logo: n8 (a magnifier whose ring is a rat's head) — decision: Use n8 in the app, launcher and splash. · status: kept - [D-214] 2026-10-25 · Host the LabRat APK on Hugging Face with a public install page — decision: Upload the 3.04 GB APK to a Hugging Face repository with its sha256 and a public install page; keep the code repository private. · why: the APK is 3.04 GB. · status: kept (2026-10-25 15:3x-15:4x (D-218): APK hosted on Hugging Face; the code repository was made public (labrat-boar-fork).) - [D-215] 2026-10-25 · Thesis restated: small models need external knowledge; a harness built around a small model's real capabilities, tuned for the app user — decision: State the thesis as: small models need external knowledge; we learned that and built a harness around a small model's real capabilities, tuned for the app user's experience; the conditional closing-test numbers are its evidence and honest limits. · why: the closing tests support only the conditional form of the 2026-10-23 sentence; the restated thesis describes what was built and learned. · status: kept - [D-216] 2026-10-25 · Publish LabRat's numbers, keep the app maintainer's internal; headline = DEVICE-FINAL; b4ef94f3 desktop run = history; applyShown documented, not fixed — decision: Numbers of BoarLite/LabRat (incl. RULER-FINAL on b4ef94f3 and the coming DEVICE-FINAL rows) are public; RC1, native-engine-branch, app-pipeline-replica and MAINT numbers stay internal-pending-author; the headline is the final v0.4 ruler on the phone; the desktop b4ef94f3 run is history; Citations.applyShown is a known limitation. · why: LabRat is the author's own app; the maintainer's numbers need his consent; a fix would invalidate the measured engine. · status: kept - [D-217] 2026-10-25 · Keep the carbon-monoxide probe answer as a known limitation; no health-gate change before the submission — decision: List the probe in the submission's limitations; no engine change. · why: an engine change before the submission would make the measured engine differ from the one installed. · status: kept - [D-218] 2026-10-25 · Make the LabRat repository public as labrat-boar-fork — decision: Rename the repository labrat-boar-fork and make it public with README, results, pack provenance and licence. · why: judges and readers can check the app and rebuild its bundle from public URLs. · status: kept - [D-219] 2026-10-25 · Submission announcement: one post with the 60 s video, no thread — decision: One post with the video plus a short poidh text, numbers from DEVICE-FINAL. · status: kept ## Part 7 · Glossary - **BOAR** [G-boar]: An Android app that answers questions with no internet, using a small AI model and libraries of text stored on the phone. Technical: Offline research assistant: deterministic router + instant tier + hybrid retrieval (SQLite FTS5 + on-device embeddings) over knowledge packs + a local reader LLM (Qwen3-4B Q4_K_M), optional 35B MoE deep tier. - **poidh bounty #31** [G-bounty]: A public crypto bounty asking for an offline Android research app that is "more than 50 % as good as internet plus a frontier AI", on a phone with at most 12 GB of RAM, in English. Technical: Open bounty on poidh; acceptance is a public confirmation by the issuer, not a numeric benchmark. Constraints: offline, ≤ 12 GB RAM, ≤ 50 GB storage, English. - **the bar (0.50)** [G-bar]: The target: BOAR's answers should be worth at least half of what a frontier AI with web search gives. Technical: Operationalised as floor-adjusted ratio CI low ≥ 0.50 (primary target, from 2026-10-06) and raw ratio CI low > 0.50 on both V-para and v0.2 (formal rule of the 2026-10-02 analysis plan). As of 2026-10-18, not reached under the Claude judge on held-out data; the research is open and pending runs could change this (data/status.json). - **frontier reference answer** [G-reference]: For every test question, a top AI model with web search wrote one answer; it is frozen and every BOAR answer is compared with it. Technical: Claude Opus with web search, prompt reference.v1, generated once per item, hashed (sha256), never regenerated, so every run shares the same denominator. - **pairwise judge (both orders)** [G-pairwise-judge]: An AI judge reads the question and both answers side by side, scores both, then reads them again in the swapped order to cancel any bias for whichever comes first. Technical: judge.v1: Claude Opus scores both answers on three 1–5 axes and picks a winner, in A/B and B/A order; scores averaged over orders; order disagreement on the winner = tie. Links, source lists and citation markers are stripped (blinding). - **rubric (correctness, completeness, usefulness)** [G-rubric]: Each answer gets three scores from 1 to 5: is it right, is it complete, is it useful. Technical: mean-rubric = mean of the three 1–5 axes, averaged over both presentation orders. The scale has a floor of 1, which inflates ratios (see rubric floor). - **raw ratio** [G-raw-ratio]: BOAR's total score divided by the reference's total score. It flatters BOAR, because even a useless answer gets 1 out of 5. Technical: Σ mean-rubric(BOAR) / Σ mean-rubric(reference) over judged items; the pre-registered formal metric (analysis plan, 2026-10-02). - **rubric floor / always-decline baseline** [G-rubric-floor]: Because the lowest score is 1, not 0, an app that refuses every question already earns about 0.31 on the raw ratio. Technical: Always-decline baseline D = 0.306 raw (79 judged declines over 7 runs; v0.2-weighted 0.307). About 61 % of a raw 0.50 is therefore "free". - **floor-adjusted ratio** [G-floor-adjusted]: The same ratio after removing the free point from both sides: how much of the reference's value above "nothing" BOAR delivers. This is the headline number. Technical: Σ(S − 1) / Σ(R − 1) with S, R the mean-rubric scores of BOAR and the reference. Pre-registered as a companion metric (2026-10-02), headline since the 2026-10-06 audit. - **win / tie / loss and win-rate** [G-win-rate]: For each question: did the judge prefer BOAR's answer, the reference, or neither? BOAR won 0 of 51 bounty-style questions. Technical: Per-item combined winner over both orders (disagreement = tie). Reported, never gated (since 2026-10-10). 0 wins on every Claude-judged row of V-para and v0.2; isolated wins on v0.1 dev only. - **formal raw rule vs primary target** [G-formal-rule]: There are two ways to read the bar; the stricter one (floor-adjusted) is the one the project leads with, and as of 2026-10-18 it has not been met. Technical: Formal rule (plan 2026-10-02): raw CI low > 0.50 on both sets — met only by the deep tier with thinking on dev data. Primary target (2026-10-06): floor-adjusted CI low ≥ 0.50 on V-para — met by no configuration as of 2026-10-18 (best 0.436, CI high 0.472). - **L1 / L2 judges (triage and confirmation)** [G-l1-l2]: A cheap AI judge screens ideas first; the expensive judge only confirms promising ones and produces the published numbers. Technical: L1 = Claude Haiku 5.5-low (replaced Sonnet-low on 2026-10-07), N = 4 shards, both orders, on 20-item slices, paired vs a named control; never an absolute L1 ratio. L2 = Claude Opus judge.v1, both orders, two seeds, for confirmations and claim numbers. Haiku compresses gaps (0.70–0.95× Opus on pack arms). - **near-miss rule** [G-near-miss]: If the cheap judge lands very close to a decision line, the expensive judge decides instead. Technical: A Haiku Δ within 0.01 of a criterion threshold is re-judged with Opus on the same ids before any verdict. - **cross-family judge (XFAM)** [G-xfam]: The same answers were re-scored by an AI judge from a different company, to see whether a Claude judge treats Claude-written references kindly or harshly. Technical: Post-hoc re-judge of existing traces with OpenAI gpt-5.6-sol (via the Codex CLI), same rubric, orders and blinding (408 calls V-para, 660 v0.2). Scores ≈ +0.10 higher on both sets, ranking unchanged (Spearman 0.88–0.97), win-rate 0.00–0.06. Not pre-registered. - **lower-judge rule** [G-lower-judge]: When two judges disagree, the project publishes the lower (less flattering) number as the headline. Technical: Claude Opus stays the headline judge because it is the stricter family; cross-family numbers are printed beside it, labelled post-hoc, and the judge family is named next to every ratio. - **judge family / self-preference** [G-judge-family]: The questions, the reference answers and the judge all came from the same AI company, which may bias the scores. Technical: Claude wrote v0.2 and V-para items and the references, and judged. Self-preference direction is measured only by one post-hoc cross-family re-judge. v0.3 is to be written outside the Claude family. - **bootstrap confidence interval** [G-bootstrap]: Resample the questions thousands of times to see how much the score could move by chance; the middle 95 % is the confidence interval. Technical: Stratified two-level bootstrap: items resampled within category, then runs within item; 2,000 resamples; paired for differences. - **common-mode CI** [G-common-mode]: A wider interval that also counts run-to-run randomness, not just which questions were asked. Technical: CI_cm = point ± 1.96·sqrt(SE_item² + (σ_run/√seeds)²), σ_run raw 0.020 / floor-adjusted 0.025 on V-para (SEEDVAR). Used for deep-tier rows since 2026-10-11. - **seed and seed noise** [G-seed-noise]: Running the same setup again with a different random seed gives different answers and a different score; that variation is seed noise. Technical: Seed = sampling seed of answer generation. Same deep configuration, s42 vs s43: Δ −0.039 (−0.072…−0.011) raw on V-para. Rule since 2026-10-16: a one-seed deep-tier Δ < 0.05 on V-para is noise (was 0.03). - **judge noise (test-retest)** [G-judge-noise]: Re-judging the exact same answers changes the score by almost nothing. Technical: σ_judge ≈ 0.003 raw / 0.004 floor-adjusted, per-item r 0.96, winner 51/51 (Opus, 102 calls): an order of magnitude below seed noise. - **MDE (minimum detectable effect)** [G-mde]: The smallest improvement a test of a given size can reliably tell apart from noise. "Not detected" below it does not mean "no effect". Technical: House MDEs: v0.2 one seed ≈ 0.025 raw (110 ids); V-para two seeds ≈ 0.03, one seed ≈ 0.06; 20-item slice ≈ 0.08; v0.1 EN 57 one seed ≈ 0.055. - **paired comparison** [G-paired]: Two versions are compared on exactly the same questions, so question difficulty cancels out. Technical: Per-item differences with a paired bootstrap CI; the default reading for every arm (Δ = arm − named control). - **winner's curse** [G-winners-curse]: If you pick the best of many setups on the same questions, it will look a bit better than it really is. Technical: The recommended deep configuration was the best of ≥ 8 deep configurations on V-para; expect a ≈ 0.01–0.02 discount on new data. - **frozen criterion (pre-registration)** [G-frozen-criterion]: Before running an experiment, the pass/fail rule is written down and fingerprinted, so it cannot be bent to fit the result afterwards. Technical: Criterion file with the decision rule, sets, n, seeds, judge and (since 2026-10-10) a Power block, hashed with sha256 before any deciding run; changes only as dated addenda. 65/65 hashes verified on 2026-10-10; an outside critic matched 16 of 17 on 2026-10-16. - **Power / Ceiling / Budget / Transfer blocks** [G-rule29]: Required parts of every experiment plan: how big an effect is expected, the best it could possibly do, what it may cost, and where it must also work. Technical: Power: expected Δ with source, n, seeds, MDE (2026-10-10). Ceiling: 0-call upper bound; launch only if ≥ 2×MDE. Budget: smoke, wall-clock caps, kill points. Transfer: a dev win must also show on the bounty-shaped set (Rule 29, 2026-10-14). - **arm (experiment)** [G-arm]: One experiment: one change compared against a fixed control, with one frozen rule and one verdict. Technical: Directory arms// with criterion (+ sha256), patch, traces, aggregates and a verdict report. 159 verified arm records by 2026-10-13. - **lever and verdict labels** [G-lever]: A "lever" is a change that measurably improves quality. Each experiment ends with a label saying whether it was one. Technical: LEVER = CI low > 0 and Δ ≥ +0.03; SMALL = real but below +0.03; NO = rule failed or CI excludes the hoped-for gain; "not detected at MDE x (n, seeds)" = CI straddles 0 below power; CHEAPER-PARITY = equal quality at lower cost. - **step-0 gate (0-call gate)** [G-step0]: A free, automatic check an idea must pass before any expensive AI judging; most ideas died here. Technical: Deterministic gate with 0 judge calls (retrieval replay, coverage counts, byte-identity), e.g. "fact chunk in top-8 ≥ bar". Of 101 closed arms counted on 2026-10-13, 54 closed at 0 judge calls. - **judge funnel (L0 → L1 → L2)** [G-funnel]: Cheap checks first, expensive judging last. Technical: L0 deterministic replay/unit gates → L1 Haiku on a random stratified 20-item slice → L2 Opus, two seeds, full set; held-out sets only by script, totals only. - **random20 slice** [G-random20]: A fixed random sample of 20 questions used only to stop hopeless ideas early, never to measure an effect. Technical: Stratified random 20-id slices (seed 7) replaced the "20 hardest items" signal20 slice on 2026-10-10; futility screen only (one n = 20 Haiku pass carries ± 0.03–0.04 noise). - **oracle / ceiling cell** [G-oracle]: A deliberately "cheating" run that hands the model the right fact, to measure the most any fix in that area could ever gain. Technical: Inject the item's known fact chunk as an extra kept passage: +0.095 (+0.070…+0.122) on the 180 pack self-tests (Opus) → headroom is retrieval reach, not reading. - **closed registry (CLOSED.md)** [G-closed-registry]: A public list of every idea that did not work, with how strong the test was and what new evidence would justify trying again. Technical: One row per closed lever family: arm(s), set, n, seeds, judge, result, MDE, label and a "would reopen it" line. 76 public rows (R1–R76); 98 rows privately by 2026-10-16. - **exposure ledger** [G-exposure-ledger]: A log with one line for every time a held-out question set was run or read, so readers can judge how clean each number is. Technical: 126 dated rows (55 v0.2-only, 33 V-para-only, 19 both); determines each set's status ruling (SET-STATUS). - **held-out (blind) set** [G-held-out]: Exam questions that the people (and agents) building the app never see; the only fair test. Technical: Item text stored outside builder-readable paths; run only by script, aggregates only; every touch logged in the exposure ledger. - **burnt set** [G-burnt]: A held-out set whose questions leaked to where builders could see them; from then on it only counts as practice data. Technical: v0.2: plaintext entered the private bench repository history; 12 generation runs and 10 Opus passes → burnt → dev by ruling 2026-10-06. - **dev vs dev-style** [G-dev-style]: Dev = practice questions you may tune on. Dev-style = a set that was a fair exam once but has since been used to make decisions. Technical: V-para is held-out for the two 4B claim runs (2026-10-05/06) and dev-style for everything measured after 2026-10-06 (ruling 2026-10-09). - **bench v0.1 / v0.2 / v0.3** [G-bench-versions]: Versions of the test set: a public practice set, a former exam set now used for practice, and a future exam to be written outside the Claude family. Technical: v0.1: 60 public items (57 EN judged), references public. v0.2: 110 items, burnt → dev. v0.3: items to be written by the author with a non-Claude model; hashes public before scoring; 0 items as of 2026-10-17. - **V-para** [G-vpara]: A set of 58 questions (51 judged + 7 venue questions) written paraphrased from Vitalik Buterin's public blog posts; the set closest to the bar. Technical: Paraphrased public-figure-blog set, created 2026-10-03; held-out for the 4B claim runs, dev-style after 2026-10-06. - **180 pack self-tests** [G-selftests]: Twenty public practice questions per knowledge pack, written from the pack's own text, used to see whether a pack helps. Technical: 9 packs × 20 agent-written items whose fact sits in a known chunk; lets the retrieval chain be traced end to end. Dev data, never claim evidence. - **reference length share (LENGTHSHARE)** [G-lengthshare]: Some of the gap to the reference is just that the reference is longer. Technical: Re-judging the same answers against references condensed to ≤ 80 words: floor-adjusted 0.412 → 0.514, ≈ 17 % length / 83 % substance (1 seed, dev-style). - **cut (frozen build)** [G-cut]: A numbered, locked version of the app's answering engine. Cut 7 is the version the claim is about. Technical: Cuts 1–7 bundle the arms that passed: cut 3 = cut 2 + verdict guard v3 + arithmetic tools + snakebite text; cut 4 adds KVFIX + instant-fragment guards; cut 5 adds a no-source KV prefix + route-gate skip; cut 6 adds DETMAX; cut 7 = cut 6 + FACTS r6 (frozen 2026-10-07). - **fast tier (4B default)** [G-fast-tier]: The default answer path: a small 4-billion-parameter model reads a few retrieved passages and writes a short answer with sources. Technical: Qwen3-4B-Instruct-2507 Q4_K_M (≈ 2.5 GB, Apache-2.0) on cut 7; fits an 8 GB phone; p50 ≈ 31 s on a 12 GB-class phone. - **deep tier** [G-deep-tier]: An optional slower mode that uses a much bigger model which "thinks" before answering; the largest quality gain measured so far, but as of 2026-10-18 it has not run on a phone (a pending run). Technical: Cut 6 + Qwen3.6-35B-A3B UD-Q2_K_XL (≈ 3B active per token) + thinking budget 512: V-para floor-adjusted 0.436 (2 seeds, dev-style, Opus), desktop p50 ≈ 79 s; projected ≈ 1.9–2.8 min on a 12 GB phone. - **lite tier** [G-lite-tier]: A smaller model option for phones with 4–6 GB of RAM: faster, less accurate, behind the same safety rules. Technical: Qwen3-1.7B Q4_K_M + X2b; matches the plain 4B only on the 180 pack-matched dev items; −0.111 vs the 4B on the claim kit; thinking stays off. - **mixture of experts (MoE)** [G-moe]: A big model split into many small "expert" blocks, of which only a few work on each word, so it knows a lot but runs cheaper than its size suggests. Technical: Qwen3.6-35B-A3B: 35B total, ≈ 3B active parameters per token. 25–35B MoEs form one closed-book plateau ≈ 0.60 raw on v0.1 vs 0.487 for the 4B. - **thinking budget** [G-thinking-budget]: How many hidden "reasoning" words the model may write before answering. 512 was as good as 1024 at about half the time. Technical: Cap on reasoning tokens; budget curve on V-para floor-adjusted: 0 → 0.33, 256 → 0.39, 512 → 0.44, 1024 → 0.42; 3072 no gain (model stops ≈ 1.3k tokens). - **expert streaming (bmoe, LES)** [G-expert-streaming]: Loading only the expert blocks a word needs from storage, instead of keeping the whole big model in memory. Technical: bmoe v0.28.0 (the on-device engine) is judged-equivalent to llama-server on identical contexts. An OpenBMB llama.cpp fork (LES) runs the 35B in ≈ 2.8 GiB RSS on a desktop CPU at 14–17 tok/s, judged-equivalent; phone flash speed unmeasured. - **quantization** [G-quant]: Storing a model's numbers with fewer bits so it fits on small devices, at some cost in quality. Technical: Deep model UD-Q2_K_XL (2-bit); 3-bit gave +0.004 (not detected, ≥ +0.03 excluded). Ternary 2B/4B/9B models were closed on speed or quality. - **instant tier (deterministic answers)** [G-instant-tier]: Answers produced by plain code, without the AI model: calculators, a facts table, emergency numbers, first-aid cards. Technical: Router paths that bypass the LLM: arithmetic, unit and travel-distance calculators, nearby places, FACTS table, first-aid cards, verbatim cited health excerpts. - **FACTS table** [G-facts-table]: A small table of facts from Wikidata that answers simple questions instantly, with a citation. Technical: 761 KB, 4,233 Wikidata entities; r6: precision 90–97 %, false positives 0/100 blind, recall 60–70 % (FAIL vs the frozen 75 %, shipped by decision in cut 7); fires 0 times on the held-out sets. - **emergency detector (DETMAX)** [G-detector]: A small text classifier that spots emergencies and health questions so the app answers them with reviewed text instead of model guesses. Technical: 106 MB learned classifier + rules; DETMAX (cut 6) takes the max of the detector score over the conversation window and the current turn. Weights not released (base-model licence unknown). - **first-aid cards** [G-cards]: Short, fixed instructions for 32 conditions, with sources and an emergency-number line. AI-reviewed, not clinically reviewed. Technical: Reviewed card texts served when the detector fires; on the default tier the model never writes health advice. Every mention must carry "AI-reviewed, not clinically reviewed". - **HEALTHGAP / NUMGUARD** [G-healthgap]: Safety rules: questions about doses, poisoning and similar count as emergencies, and only a deterministic calculator may override a fired emergency. Technical: HEALTHGAP r2 widens the emergency predicate; NUMGUARD r3 lets only the calculator answer a numeric question the detector flagged (the lexicon guard was rejected as unsafe by review). - **verdict guard (self-contradiction guard)** [G-verdict-guard]: A check that catches answers which say "yes" and then argue "no". Technical: Deterministic repair of verdict/body contradictions; v3 shipped in cut 3 after a reviewer gate; v4 failed (1 WRONG of 2 fires). - **route gate** [G-route-gate]: A one-word decision by the model: search the libraries, or just chat. Technical: One-token classification by the 4B (chat vs search); 95 % on a dev chat slice; skipped for latency in cut 5 when safe. - **knowledge pack** [G-pack]: A downloadable offline library (one file) of articles on one topic that the app searches before answering. Technical: SQLite "format 2" (format-v3 adds chunk vectors) with per-row licence and attribution, a card, 20 public self-tests, a coverage probe and a crowding check. Licences: PD, CC0, CC BY, CC BY-SA only. 11 packs published on 2026-10-13. - **retrieval funnel: reach → keep → read** [G-reach-keep-read]: Reach: is the right passage found? Keep: does it survive the filter into the prompt? Read: does the model use it correctly? Technical: reach = fact chunk in the raw top-k; keep = grounding filter that drops off-topic/unsafe passages and compresses the rest; read = reader output given the kept context. - **crowding** [G-crowding]: Installing many libraries at once lets a big general one take the few slots the right small one needed. Technical: All packs installed vs the matching pack only: −0.019 (−0.035…−0.003) Opus on the 180 self-tests; packs above 0.40 of kept slots are user-selected only. - **HEADPASS-X2b** [G-x2b]: The one retrieval change that worked: also feed the model the passages next to the best-matching one, plus a few found by meaning, within a fixed size limit. Technical: After the keep step, append the top article's neighbour chunks + ≤ 2 dense candidates under +600 prompt tokens: +0.066 (+0.043…+0.091) Opus on the 180 (dev); inert on the claim kit. - **PSS / RSS, TTFT, prefill / decode** [G-pss]: Ways to measure an app's memory and speed: how much memory it really uses, and how long until the first word appears. Technical: PSS = proportional set size (shared pages split fairly); RSS = resident pages. TTFT = time to first token. Prefill = reading the prompt; decode = generating tokens. - **desktop proxy and device parity** [G-desktop-proxy]: The exact phone engine also runs on a Mac for fast testing; real phones confirmed that the Mac gives the same judged quality. Technical: Same engine code and tree hash on desktop and phone; parity on v0.1: 12 GB-class phone 0.529 = desktop 0.529 (Haiku); 8 GB phone 0.561 vs 0.558/0.560 (Opus). - **no-network APK** [G-no-network-apk]: A build of the app with the internet permission removed, so the phone itself blocks any network access. Technical: INTERNET permission removed from the manifest: offline by construction, not by promise. - **confident-wrong** [G-confident-wrong]: An answer that is wrong but stated with confidence, the most harmful failure; tracked separately. Technical: Count of items judged wrong with no hedge; a guard metric in many criteria (e.g. "confident-wrong not up"). - **coordinator, research lanes and independent reviewers** [G-coordinator]: The research was run by AI agents: one coordinator ran the experiments, specialist agents proposed ideas in their own areas, and separate reviewer agents checked wording and numbers. Technical: Coordinator session (Opus/Fable) owns the queue, freezes criteria and writes verdicts; lane agents (retrieval, models/latency, evaluation/safety, native performance, a runner, an on-call strategy agent) write technique cards and never run experiments; independent reviewer agents decide wording ("AI-reviewed"). - **unattended research loop and ticks** [G-loop]: For four days an AI coordinator woke up on a timer, advanced one experiment, logged it and scheduled its next wake-up, without a human. Technical: 133 ticks (28 idle), 2026-10-11 → 10-14; META tick every 8 ticks re-ranks the backlog; LOOP v2 (2026-10-14) moved queueing to machine-side runners. - **handoff** [G-handoff]: A file that lets a fresh AI session take over the coordinator role with no memory of the previous one. Technical: Dated HANDOFF-*.md with a §0 to paste into a new session: state, running jobs, rules, next steps. - **claim text** [G-claim]: The text for the bounty that says, with numbers and caveats, what BOAR does and does not achieve; it follows the current standing and is posted at closure. Technical: Claim draft v4.2.4 (2026-10-17): "BOAR is not shown to be half as good as frontier AI with web"; judge family named next to every ratio; leads with win-rate. - **submitted app vs research engine** [G-native-app]: The app actually submitted to the bounty is the app maintainer's own rewrite; the research numbers describe the engine it partly ports. Technical: Submission vehicle = the app maintainer's native Kotlin app (from 2026-10-15); it ports retrieval and prompts but not the health path or deterministic answers. All research numbers are for the TypeScript research engine (cut 7). - **ruler (bench v0.4)** [G-ruler-v04]: A fixed set of 75 test questions, frozen on 2026-10-20 before any system answered them, meant to give the final score of the submitted app. Technical: 75 items = 71 headline + 4 anchor questions taken from Vitalik Buterin's public post (scored outside the headline); written half by Claude and half by OpenAI gpt-5.6-sol via the Codex CLI, cross-fact-checked; sha f9ad2f2f. Not written outside the Claude family (a stated deviation). First exposed to two engines on 2026-10-24. - **ruler spec (RULER-SPEC v1)** [G-ruler-spec]: The written procedure for scoring the ruler: which references, judges, metrics and gates, frozen before scoring. Technical: r2 (2026-10-20) was a draft; r3 (2026-10-21 20:48, sha 0fac2286) is the first frozen spec (judge.v1 + DECLINE2STAGE, verdict arm A∪, two judge families); r4 (21:18, sha 82772e33) adds the information-available split (FACT-AVAIL) and ORACLE / ORACLE-0 rows. Spec files are private. - **U-total** [G-u-total]: The ruler's headline number: how much of a reference answer's useful content the app delivers, over all 71 headline questions. Technical: Ratio of sums: Σ(app mass) / Σ(reference mass), where each mass is the mean of correctness, completeness and usefulness minus 1 (judge.v1, 1-5 scale), so the rubric floor maps to 0; detected non-answers are set to 0 by DECLINE2STAGE; health items weigh 0.5 (weight-1 printed beside). Verdict statistic = the lower of the two judge families per bootstrap draw; "meets the bar" needs CI low > 0.50. A different metric from the judge.v1 V-para / v0.2 ratios: never compare across. - **DECLINE2STAGE** [G-decline2stage]: A first check that gives zero to answers that do not try to answer, before the usual judge scores the rest. Technical: Stage 1: a classifier asks whether the answer attempts an answer; if not, U = 0; otherwise judge.v1 scores it unchanged. Adopted as RULING R1 (2026-10-21) after the decline arm was invalid under judge.v1 (a bare refusal scored U 0.016-0.058); a coordinator override of a frozen FAIL, accepted with conditions by the spec reviewer. The decline offset D 0.058 is kept as a sensitivity row (U_net). - **FACT-AVAIL (information-available split)** [G-fact-avail]: Labels saying, for each ruler question, whether the answer is actually written in the installed knowledge packs. Technical: Pre-registered per item: AVAILABLE (≥ 0.5 of checklist points stated in the persona packs) / PARTIAL / NONE; a lower bound. On the 71 headline items: 19 / 25 / 27 (strict 15 / 23 / 33). Labels are quarantined until the result card; counts only. - **A∪ (verdict arm)** [G-a-union]: The app with every persona knowledge pack installed at once, used for the verdict so no question gets a hand-picked pack. Technical: All six persona packs installed once for all items (wikivoyage, visa, crypto, medlineplus, firstaid, factbook; ≈ 435 MB); per-family arm A is a labelled row only, because routing packs by topic would be oracle routing. - **ORACLE / ORACLE-0** [G-oracle-arms]: The same model answering the same questions with and without the supporting passages handed to it. Technical: ORACLE − ORACLE-0 on the same items = the information effect (what delivering the right passages is worth). Defined in spec r4; not run as of 2026-10-24. - **outcome labels (CWS, correct-any, WU, WW, HU)** [G-outcome-labels]: Each answer is put into one class: right with the right source shown, right from memory, wrong with no warning, wrong with a warning, or an honest "unknown". Technical: Shares of n items labelled by Claude Haiku 5.5-low under the UXMATRIX protocol: CWS = correct with the right source shown; CFM = correct from memory; correct-any = CWS + CFM; WC/WU = wrong with no warning; WW = wrong with the warning line; HU = honest unknown; RW = refused wrongly. Not a ratio to a reference: never compare with the judge.v1 ratio or with U-total. - **delivery (kept / delivered fact)** [G-delivery]: Whether the passage that holds the answer actually reaches the small model's prompt after search and compression. Technical: Counted at chunk level from 2026-10-22: the key fact's quote sits inside one kept chunk (article-level counting, "a chunk of the right article", overstated delivery by 8-13 facts). Article-level and chunk-level counts are never pooled. - **conditional thesis** [G-conditional-thesis]: The author's closing claim, in the form the tests support: the app is right about two times in three when the right passage reaches the model. Technical: Stated by the author on 2026-10-23 as "right whenever the content is in a knowledge pack"; PACKPROOF supports only the conditional form: correct-any 0.65 when the item's own article is kept vs 0.28 when not, with the pack installed 0.48 vs 0.26 without (n 180 pack-written self-tests, Claude Haiku labels, one seed, dev); no pack reached the 0.75 bar. - **pack self-test** [G-pack-self-test]: Questions written from a pack's own text, used to check that pack; they favour the pack by construction. Technical: An on-topic upper bound, not casual traffic: self-test gains (e.g. PACKPROOF +0.23 correct-any) did not transfer to real dev items in the same week (delivered-vs-not +0.001 between items). - **global-IDF merge (gidf)** [G-gidf]: Re-scoring each pack's keyword hits with word rarity computed across all installed packs, so small packs compete fairly with big ones. Technical: Part of the adopted search stack SEFIX-124 (gidf + SEFIX F1/F2/F4); chunk-level delivered MAIN facts 28 → 42 of 92; answer gain +0.013 (−0.008…+0.035, Claude Haiku, 2 seeds), not a lever. - **SEFIX-124** [G-sefix]: The adopted search fixes for question-and-answer packs. Technical: Global-IDF merge plus three lexical fixes (F1 lead-vector thread pick, F2 domain words kept, F4 relaxed keep rule); F3 (a health no-source fallback) was withdrawn because it made safety probes worse (unsafe 9 → 13 of 65). - **ENGINE-RC** [G-engine-rc]: The closed research engine assembled as one build for validation. Technical: SEFIX-124 search + W78 reading-cost stack + shaped factbook + reworded warning banner, Succinct style, Deep off; passed every no-LLM suite on 2026-10-24 (desktop only). The research engine, not the submitted app. - **W78 stack** [G-w78]: A compact search index stored inside each pack, so a question reads far less data from storage. Technical: 1-bit sign sidecar + int8 rescore + IVF lists stored in the pack: 125.09 → 21.26 MB read per question on a desktop; phone effect unmeasured. - **pack shaping (shaped factbook)** [G-pack-shaping]: Rewriting table-like pack text into short sentences that name their subject, without adding facts. Technical: Shaped factbook v2: fact chunk kept 7 → 15 of 20; correct 5 → 10 of 20; Claude Opus U 0.316 → 0.436 (+0.121, +0.030…+0.221; 2 seeds, n 20, dev); NOT-CONFIRMED by its frozen rule (one new warned confident error). In ENGINE-RC; publication of the pack waits for the author. - **warning line** [G-warning-line]: A line the engine puts before an answer its kept passages do not support, so the reader knows it comes from model memory. Technical: Two texts: "Not found in this phone's offline library…" when nothing is kept, and "The passages found don't support this answer…" when passages are kept but every citation is stripped. A fixed-banner bug showed "Not found" on 37 of 97 answers that kept passages; fixed (0 after the fix). - **refuse by default / show anyway** [G-refuse-by-default]: With no source found, the app declines and offers a button to see the model's unsourced answer behind a warning. Technical: Author decision 2026-10-20 20:24; narrowed on 2026-10-21: show-anyway only behind a second confirmation and never on health; validated for the 1.7B / Gemma only. The closed app (2026-10-24) keeps the warning line and shows sources. - **confident error (CE, CE-unwarned)** [G-confident-error]: A wrong answer shown to the user without a warning. Technical: An answered item with judge.v1 mean correctness ≤ 1.5 of 5, not refused; "unwarned" = shown without the warning line. Counted per 100 questions under a named mode, model and judge; judge retest noise ± 1.3 per 100. - **GATE v2 (two-stage screen)** [G-gate-v2]: A cheap first test on the items the current build gets wrong plus items it gets right, with a "do nothing" run to see how much changes by chance. Technical: Net = problems fixed − new visible confident errors; GO iff net ≥ max null-arm net + 2 (null changes such as a new seed fix 23-31 % of problems); stage 2 = full MAIN 97. - **MAIN 97 / CV5 51** [G-main-97]: The two development sets used for most screens after 2026-10-18. Technical: MAIN 97 = 57 public v0.1 items + 40 cooking/money pack self-tests (dev); CV5 51 = 51 v0.1 ids used for Claude Opus ladders (dev). Neither is held-out. - **BoarLite** [G-boarlite]: The author's own closed, offline chat app, declared the bounty submission on 2026-10-24; renamed LabRat on 2026-10-25. Technical: No internet permission; the full K13 pack kit bundled in one APK (≈ 3.04 GB); Qwen3-1.7B fixed; Succinct only (no Deep, Detailed or length option); factbook v1 (the shaped v2 was dropped on 2026-10-24 18:03); a fixed line under every answer instead of a warning trigger; health gate B+FU; QuickMath; search port ON_NO_HEALTH. Final candidate b4ef94f3 as of 2026-10-24 19:40 (UI-only commits after it), smoke-tested on a Pixel 8a; its desktop-twin ruler scores were published by the author's decision of 2026-10-25 as history (F-2215; before the Rat persona line). Renamed LabRat on 2026-10-25 (7788924a), which also adds the Rat persona line to the grounding prompt after the ruler run on b4ef94f3; product cleanup the same night (Expo/React Native tree, Portuguese, PIN and long-chat summary removed). - **K13 kit** [G-k13]: The default set of 13 knowledge packs used in every closing test. Technical: Includes the Wikipedia packs (kept after NOWIKI: removing them lost 9 of 42 delivered MAIN facts and 10 public v0.1 items). - **desktop twin** [G-desktop-twin]: Running an app's engine on a desktop harness instead of the phone. Technical: Counts as dev or parity evidence, never as a verdict; every closing test and both first ruler v0.4 runs (2026-10-24) were desktop twins. - **health gate B+FU** [G-health-gate-bfu]: BoarLite's rule for health questions: when no trusted health source is found, it shows a fixed safe text with the emergency number instead of a model answer. Technical: Predicate = the app's lexical health test OR 11 lexical rules (B: drugs, doses, vitals, child fever, symptoms, poisoning, alcohol, pregnancy, child dosing, trauma, altitude, diving) OR a short follow-up of a health turn (FU); runs after retrieval; health sources = firstaid, MedlinePlus, health Q&A packs. Final candidate: 41 of 65 safety probes fixed text, 0 new false fires on 251 non-health dev items; rules fitted on the probes. - **QuickMath** [G-quickmath]: Unit conversions and simple arithmetic are worked out by code, with no model and no sources. Technical: BoarLite 43b50410: runs at the start of the answer pipeline; never on health-gated turns or turns after a health turn; exposure questions go to the health gate instead. - **grounded-dose rule** [G-grounded-dose-rule]: A health answer counts as safe only if it is the fixed safe text or every dose it states appears in a source shown with it. Technical: Frozen 2026-10-24 19:1x; scorer extracts figures with units, normalises them and matches them numerically against the shown passages and the question; replaced kept-list proxies. Known limit: a grounded figure can be a toxic threshold read as a dose (arm B). - **ON_NO_HEALTH** [G-on-no-health]: BoarLite uses the improved search for every question except health questions, which keep the old, more cautious search. Technical: SearchPort mode: SEFIX-124 delivery + head pass + X2b appends compressed within 1.2× (SearchPort.fit), off on turns the health predicate fires on (incl. follow-ups). Delivery MAIN 26 → 40; its frozen criterion failed by the letter (S1 6 vs 4 ungrounded figures) and it was merged under the author's prior condition. - **fixed line** [G-fixed-line]: One short line under every BoarLite answer: "May contain errors; check the source." Technical: Replaced the ported per-answer warning trigger (D4) on 2026-10-24 18:03; drawn outside the answer text, not on the fixed health text or computed answers. - **LabRat** [G-labrat]: The submitted offline chat app's name from 2026-10-25; it was called BoarLite before. Technical: Renamed at the app maintainer's request in commit 7788924a; same package id and bundled knowledge as BoarLite; the grounding prompt's identity line names the assistant Rat (added after the RULER-FINAL run on b4ef94f3). Measured on the phone by DEVICE-FINAL. - **Rat** [G-rat]: The name LabRat's assistant uses for itself in its answers. Technical: Set by one identity line in the grounding prompt ("You are Rat, the assistant in LabRat…", 7788924a), replacing "You are BOAR…"; the only prompt change between the ruler engine b4ef94f3 and LabRat 7788924a named in the DEVICE-FINAL criterion. - **DEVICE-FINAL** [G-device-final]: The last benchmark run: the submitted app answers the test sets on the phone itself, overnight. Technical: Frozen criterion 2026-10-25 03:3x: LabRat 7788924a on a Pixel 8a; M1 ruler v0.4 × seeds 42/43/44 (Claude Opus all seeds, OpenAI gpt-5.6-sol seed 42), M2 65 safety probes, M3 140 on-K13 self-tests, M4 UXMATRIX 60, M5 speed/memory/offline. Claim numbers = M1 + M5. Running as of the 2026-10-25 update; no results recorded. ## Part 8 · Facts The canonical facts F-001 to F-046 and every other fact cited above, in id order, except the rows already given in full in Part 2. All rows: https://r4topunk.github.io/boar-research/data/facts.json. - [F-003] A system that declines every question already scores about 0.31 on the raw ratio, because the rubric never goes below 1 of 5. always-decline raw ratio D = 0.306 (v0.2-weighted 0.307) (judge.v1 ratio) · set v0.1 night runs (dev, development data) · judge Claude Opus (judge.v1, both orders) · n 79 judged declines over 7 runs · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: This is why the floor-adjusted ratio, not the raw ratio, is the headline. - [F-005] Swapping in the 35B mixture-of-experts deep tier (no thinking) raises the bounty-style score. raw 0.467 (0.439–0.494); floor-adjusted 0.329 (0.294–0.362); win 0/51; desktop p50 ≈ 33 s (judge.v1 ratio) · set V-para (dev-style (measured after 2026-10-06), development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI 95 % item bootstrap · date 2026-10-08 · caveat: Desktop only; the deep tier has never run on a phone. - [F-006] Deep tier (no thinking) on v0.2. raw 0.568 (0.536–0.602); floor-adjusted 0.457 (0.417–0.500); win 0/110 (judge.v1 ratio) · set v0.2 (dev (burnt), development data) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 2 · CI 95 % item bootstrap · date 2026-10-08 - [F-007] Letting the deep tier think for up to 1024 tokens before answering. raw 0.542 (0.513–0.571); floor-adjusted 0.422 (0.385–0.459); win 0/51; desktop p50 ≈ 143 s (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI 95 % item bootstrap · date 2026-10-08 - [F-008] Thinking is the largest single measured gain on the bounty-style set. thinking (1024) vs no thinking: raw +0.075 (+0.040…+0.110); floor-adjusted +0.093 (+0.050…+0.137) (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI paired 95 % bootstrap · date 2026-10-08 - [F-010] A 512-token thinking budget matches 1024 in quality at about half the time. 512 vs 1024 on V-para: raw +0.011 (−0.010…+0.031), floor-adjusted +0.014 (−0.012…+0.039), ×0.55 time; on v0.2 −0.005 (−0.020…+0.009), ×0.62 time (judge.v1 ratio) · set V-para; v0.2 (dev-style / dev, development data) · judge Claude Opus (judge.v1, both orders) · n 51 / 110 · seeds 2 (V-para) / 1 (v0.2) · CI paired 95 % bootstrap · date 2026-10-09 · caveat: Verdict label: CHEAPER-PARITY (an equivalence, not a gain). - [F-011] A 256-token budget is cheaper but measurably worse. raw 0.514 (0.483–0.544); floor-adjusted 0.387 (0.349–0.426); vs 1024 raw −0.028 (−0.056…−0.003); p50 ≈ 55 s (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI paired 95 % bootstrap · date 2026-10-09 · caveat: Keeps ≈ 63 % of the thinking gain at 40 % of the time. - [F-012] Three seeds of the recommended deep configuration show how much one run can move. raw mean 0.550, SD 0.020 (0.533 / 0.572 / 0.544); floor-adjusted mean 0.432, SD 0.025 (0.412 / 0.460 / 0.425); 0 wins each seed (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 3 (42–44) · CI pooled CI UNKNOWN (aggregate file empty) · date 2026-10-10 - [F-013] Recommended deep configuration on v0.2: the first point above 0.50 floor-adjusted on any set, with a CI that straddles 0.50. raw 0.608 (0.575–0.641); floor-adjusted 0.508 (0.467–0.550); win/tie/loss 0/1/109; desktop p50 ≈ 82 s (judge.v1 ratio) · set v0.2 (dev (burnt), development data) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 1 (42) · CI 95 % item bootstrap · date 2026-10-09 · caveat: One seed, development data, desktop only. - [F-014] Deep tier with 1024-token thinking on v0.2. raw 0.612 (0.581–0.644); floor-adjusted 0.514 (0.475–0.554); 0 wins (judge.v1 ratio) · set v0.2 (dev (burnt), development data) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 2 · CI 95 % item bootstrap · date 2026-10-09 - [F-015] Anchor: a small frontier model with no web access (Claude Haiku 5.5) on the bounty-style set. raw 0.588 (0.559–0.615); floor-adjusted 0.48 (CI unknown) (judge.v1 ratio) · set V-para (held-out at the time (2026-10-05)) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 1 · CI raw 95 % item bootstrap; floor-adjusted CI never computed · date 2026-10-05 · caveat: The best deep configuration (0.436, CI high 0.472) does not reach this anchor either. - [F-016] Win-rate against the frontier reference: zero on both bounty-relevant sets under the Claude judge; a few isolated wins exist only on the public dev set. 0 wins on every row and seed of V-para and v0.2 (Opus); v0.1 dev: 1/57 under Opus for the deep R450, LES and 4B rows; 2/57 under Haiku on the 12 GB-class phone run · set V-para; v0.2; v0.1 (dev-style / dev / public dev, development data) · judge Claude Opus; Claude Haiku L1 (phone run) · n 51 / 110 / 57 · seeds all · CI not computed · date 2026-10-14 · caveat: Public text that says "0 everywhere" (boar-research README.md:42, boar-bench TRACE-FORMAT.md:70) overstates the scope. - [F-019] Re-judging the exact same answers barely moves the score: judge noise is negligible next to seed noise. σ_judge ≈ 0.003 raw / 0.004 floor-adjusted; per-item r 0.96; same winner 51/51 (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 (102 Opus calls) · seeds 1 (re-judge of s42) · CI n 51; 102 calls · date 2026-10-11 - [F-020] Seed noise: the same deep configuration run with two random seeds differs by more than many tested "levers". s42 vs s43, same configuration: Δ −0.039 (−0.072…−0.011) raw; house rule since 2026-10-16: a one-seed deep-tier Δ below 0.05 on V-para is seed noise (raised from 0.03) (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI paired 95 % bootstrap · date 2026-10-16 · caveat: The public boar-research README.md:77 still says 0.03. - [F-021] Part of the gap to the reference is length: shortening the references to 80 words raises the deep tier's score, mostly through completeness and usefulness. deep + think 512 floor-adjusted 0.412 → 0.514 (0.456–0.578) with ≤ 80-word references; paired +0.102 (+0.068…+0.144); ≈ 17 % of the gap is length, 83 % substance (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 1 (42) · CI paired 95 % bootstrap · date 2026-10-11 · caveat: Diagnostic with altered references; not a bar reading. - [F-022] On correctness alone the deep tier is much closer to the reference than on the full rubric. floor-adjusted correctness-only: V-para 0.660 (0.608–0.712); v0.2 0.743 (0.681–0.805) (judge.v1 ratio) · set V-para; v0.2 (dev-style / dev, development data) · judge Claude Opus (judge.v1, both orders) · n 51 / 110 · seeds 2 / 1 · CI 95 % item bootstrap · date 2026-10-10 · caveat: Single-axis diagnostic, not a bar reading. - [F-023] On a real 12 GB-class phone, in airplane mode, the frozen build gives the same judged quality as on the desktop. phone raw 0.529 (0.481–0.575) vs desktop 0.529; floor-adjusted 0.403 vs 0.402; paired −0.000 (−0.026…+0.028); 72/72 answered; fast-tier p50 30.7 s; app memory (PSS) peak 5.88 GB during model load, then ≤ 4.23 GB (judge.v1 ratio) · set v0.1 (public dev, development data) · judge Claude Haiku 5.5-low (L1 triage judge, N = 4, both orders) · n 57 · seeds 1 run · CI paired 95 % bootstrap · date 2026-10-14 · caveat: Haiku-judged level; not comparable with Opus-judged ratios. The device has 10.7 GiB usable RAM. - [F-024] Earlier, on an 8 GB phone, cut 6 also matched the desktop. phone 0.561 (0.507–0.617) vs desktop 0.558 / 0.560; 73/73 answered; PSS max 4.41 GiB · set v0.1 (public dev, development data) · judge Claude Opus (judge.v1-med) · n 57 · seeds 1 run · CI 95 % item bootstrap · date 2026-10-06 · caveat: Opus-judged; not comparable in level with the Haiku-judged 12 GB-class run. - [F-025] One matching knowledge pack helps the small model a little on questions written from that pack. Opus +0.042 (+0.024…+0.062); Haiku +0.032 (+0.016…+0.050) · set 180 pack self-tests (dev (agent-written from pack text), development data) · judge Claude Opus (L2) and Claude Haiku (L1) · n 180 · seeds 1 · CI paired 95 % bootstrap · date 2026-10-13 · caveat: Pack-matched dev questions, not a bounty-bar measurement. - [F-026] Installing all packs at once crowds out the right passage and costs a little quality. Opus −0.019 (−0.035…−0.003); Haiku −0.016 (−0.033…−0.001) · set 180 pack self-tests (dev, development data) · judge Claude Opus (L2) and Claude Haiku (L1) · n 180 · seeds 1 · CI paired 95 % bootstrap · date 2026-10-13 - [F-027] Ceiling test: handing the model the exact fact-bearing chunk shows the headroom is retrieval, not reading. inject fact chunk (ADD) +0.095 (+0.070…+0.122); fact chunk only (ONLY) +0.115 (+0.088…+0.142) · set 180 pack self-tests (dev, development data) · judge Claude Opus (judge.v1, both orders) · n 180 · seeds 1 · CI paired 95 % bootstrap · date 2026-10-13 · caveat: Oracle (cheating) cell: an upper bound, not a deployable result. - [F-028] The one retrieval change that converted: append the retrieved article's neighbouring chunks plus a few vector-search candidates, under a 600-token budget (HEADPASS-X2b). Opus +0.066 (+0.043…+0.091); Haiku +0.056; fact chunk present in context 35 → 102 of 180 · set 180 pack self-tests (dev (designed on the same set), development data) · judge Claude Opus (judge.v1, both orders) · n 180 · seeds 1 · CI paired 95 % bootstrap · date 2026-10-14 · caveat: LEVER on pack-matched dev data only; inert on the claim kit (ceiling ≈ 0.011) and not a bounty-bar lever. - [F-029] The 35B deep tier gains nothing measurable from packs; its advantage over the 4B with the same pack is model knowledge. deep with vs without pack Δ −0.002 (−0.020…+0.016); deep − 4B with the same pack +0.054 (+0.034…+0.073) · set 180 pack self-tests (dev, development data) · judge Claude Haiku 5.5-low (L1 triage judge, N = 4, both orders) · n 180 · seeds 1 · CI paired 95 % bootstrap · date 2026-10-13 - [F-030] Long conversations do not degrade answers up to turn 15. turns 6–15, in-session vs standalone: 1.136 (0.993–1.320); self-contained items 1.017 (0.950–1.095); latency ≈ 22 s vs 8 s at turns 14–15 · set LONGSESSION dev sessions (dev (agent-written), development data) · judge Claude Haiku 5.5-low (L1 triage judge, N = 4, both orders) · n 80 agent-written questions · seeds 1 · CI 95 % bootstrap · date 2026-10-14 · caveat: The reference here is BOAR's own standalone answer, not the frontier reference: a ratio of ratios, not a bar number. - [F-031] Four days of an unattended agent research loop produced one dev-only lever. 133 ticks (28 idle), 2026-10-11 12:10 → 10-14 19:09; ≈ 10.5k judge calls (Opus 4,851 + Haiku 5,660) plus 1,491 Haiku writer/generation calls; Mac mini 44.4 h + MacBook 18.1 h of generation; ≈ 31 h avoidable · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 - [F-032] Early ceiling test on the public dev set: even the gold passage leaves the 4B below a no-web frontier model. raw: closed-book 0.487; full engine 0.507; with the gold passage 0.606; Claude Haiku without web 0.653 (judge.v1 ratio) · set v0.1 (EN) (public dev, development data) · judge Claude Opus (judge.v1, both orders) · n 57 · seeds 1 · CI n 57 · date 2026-10-04 · caveat: Raw ratios only; floor-adjusted values were not computed. - [F-033] The first deep-tier test was the first lever to beat noise. deep35 vs 4B: 0.565 vs 0.505, Δ +0.060 (+0.021…+0.101) raw (judge.v1 ratio) · set v0.1 (public dev, development data) · judge Claude Opus (judge.v1, both orders) · n 57 · seeds 2 · CI paired 95 % bootstrap · date 2026-10-02 · caveat: Raw ratio on the public dev set. - [F-034] Mixture-of-experts models in the 25–35B class form one closed-book plateau, well above the 4B. closed-book raw: Gemma 4 26B-A4B 0.613; Qwen3.5-35B-A3B 0.603; Qwen3.6-35B-A3B 0.596; vs 4B 0.487 (judge.v1 ratio) · set v0.1 (public dev, development data) · judge Claude Opus (judge.v1, both orders) · n 57 · seeds 1 · CI 1 seed; n 57 · date 2026-10-07 · caveat: Screen differences between the MoE models are not detected at MDE 0.055. - [F-035] Projected phone time for the recommended deep configuration (never measured on a phone). ≈ 1.9–2.8 min per answer at thinking budget 512 on a 12 GB phone CPU (third-party public decode rates of 4–6 tok/s + BOAR arithmetic); the 2026-10-09 estimate of 3.5–5 min was for budget 1024 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI projection · date 2026-10-12 · caveat: Third-party rates, not a BOAR measurement. - [F-036] An expert-streaming engine runs the 35B model in under 3 GiB of RAM on a desktop CPU with judged-equivalent answers. judged 0.625 vs llama-server 0.634, paired −0.010 (−0.042…+0.018); RSS 2.75–2.81 GiB; 14–17 tok/s on the Mac mini CPU; 10/10 greedy-identical vs the fork base · set v0.1 (public dev, development data) · judge Claude Opus (judge.v1, both orders) · n 57 (R450 contexts) · seeds 1 · CI paired 95 % bootstrap · date 2026-10-16 · caveat: Desktop only, experts read from a fast SSD and page cache; phone flash speed is unmeasured (analytic bound ≈ 4–6 tok/s with compute). - [F-037] The on-device MoE engine (bmoe) is not token-identical to the desktop server but is judged equivalent on identical contexts. 0.623 vs 0.634, Δ −0.012 (−0.045…+0.020); greedy token-identical 0/10 · set v0.1 (public dev, development data) · judge Claude Opus (judge.v1, both orders) · n 57 · seeds 1 · CI paired 95 % bootstrap · date 2026-10-07 · caveat: An equivalence, not a lever: lets desktop deep-tier numbers transfer to a bmoe device plan with this caveat. - [F-039] Before the 2026-10-06 audit, a week of 4B improvements did not move the held-out score beyond noise. v0.2 raw: main 0.506 → cut 6 0.512, two seeds each, +0.006 (noise) (judge.v1 ratio) · set v0.2 (held-out at run time (burnt → dev 2026-10-06)) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 2 · CI not computed · date 2026-10-06 · caveat: Do not compare 0.519 (one seed, s42) with the two-seed 0.506; the audit's "+0.007" and QUEUE:703 use other pairings. - [F-040] Deadlines: the bounty page (swept 2026-10-13) gives a final decision by 2026-10-31, and the issuer has said there is no fixed deadline; 2026-10-20 was the team's own closing-sprint deadline. 2026-10-31 (bounty page); 2026-10-20 (closing sprint) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: UNKNOWN whether the bounty itself moved to 10-20; call 10-20 the team's closing-sprint deadline unless the author confirms. - [F-041] By 2026-10-13 the ledger audit had verified 159 experiment records from 101 arm folders. 159 arm records (closed 99, lever-adopted 20, measurement 10, tooling 11, not-decided 9, parked 4, running 1, other 5) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 · caveat: Another document counts 154; QUEUE:1089 (ledger audit v2) is the cited count. - [F-042] Frozen pass/fail criteria were re-checked against their sha256 fingerprints. 65/65 criterion hashes verified (2026-10-10); an outside critic found 16 of 17 matching (2026-10-16), and 19 of 20 in a spot-check (1 hashing-convention mismatch, 1 criterion without a sidecar) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n 65 / 17 / 20 · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: Some early freezes are attested by file mtime only. - [F-043] Negative results are kept in a public registry, each with its power and what would reopen it. 76 public rows (R1–R76) on 2026-10-14; 98 rows in the private registry by 2026-10-16 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: Rows R77–R98 are not yet public and need a scrub (they mention internal items). - [F-044] The floor-adjusted scale was pre-registered as a companion metric before any held-out result; it was promoted to headline after the audit. 2026-10-02: plan lists floor-adjusted as companion (plan sha 375ff004…); 2026-10-06: audit decision D1 makes it the headline with win/tie/loss; 2026-10-07: applied in claim draft v3.1 (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-06 · caveat: The headline promotion came after the first held-out results; all three scales are always printed. - [F-045] How the win-rate condition changed: secondary metric in the plan, briefly part of the bar in arm reports, then "reported, never gated". 2026-10-02 plan: W/T/L secondary; 2026-10-08/09 arm reports read the bar as "floor-adjusted CI low > 0.50 AND win-rate > 0"; 2026-10-10 BAR-TABLE: win-rate reported, never gated (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10 - [F-046] The cut 6 freeze was relaxed once: cut 7 added the Wikidata fact table, which changed 8 public dev answers. 2026-10-06 rule: only routing/safety fixes with v0.1 answers unchanged; 2026-10-07: cut 7 = cut 6 + FACTS r6 changed 8 v0.1 answers (0 fires on the held-out sets) · set v0.1 (public dev, development data) · judge UNKNOWN · n 8 of 57 v0.1 answers · seeds UNKNOWN · CI not computed · date 2026-10-07 - [F-047] Public post (by Vitalik Buterin) asks for an offline phone research tool ">50 % as good as internet + frontier AI"; bounty #31 is later opened on poidh, inspired by it. · date 2026-09-17 - [F-058] Earlier MoE probe: Qwen3.6-35B took 40–58 s to first token; CPU prefill was the limit. 40–58 s · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-09-29 · caveat: pre-harness desktop/phone mix - [F-059] References = Claude Opus + web search, frozen once, hashed; never regenerated (why: a fixed denominator makes runs comparable) · date 2026-09-30 → 10-02 · caveat: outcome: same refs used for every v0.1 number; regenerating = not comparable - [F-063] Before the harness, an overnight on-phone session lifted the old 61-question bounty-style score from 0.26 to 0.48. 0.26 → 0.48 · set old "v2" 61 items, BOAR's own dev set (dev, development data) · judge Claude judge, both orders vs Opus+web · n 2 · seeds UNKNOWN · CI CI 0.42–0.56, replicated in 2 runs · date 2026-09-30 · caveat: different instrument, model (1.5B), phone and set; NOT comparable to later v0.1/v0.2/V-para numbers - [F-067] References were generated by Claude Opus 5.5 with WebSearch+WebFetch, one call at a time, 5 s gap, 1 retry, prompt reference.v1. 60 refs · set v0.1 (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-09-30 → 10-02 · caveat: references are AI output, "not ground truth" - [F-071] The bounty's hardware envelope. ≤ 12 GB RAM, ≤ 50 GB total storage, offline, no Play Services, GrapheneOS-compatible · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: GrapheneOS not yet run on hardware - [F-075] Single goal = win the bounty; English only; no more work inside the maintainer's app; build a dedicated measurement app — why: focus — outcome: PT/ES lines parked; BOAR Bench built. · date 2026-10-03 - [F-091] Wave-1 packs barely cover the bounty-style set. V-para absent-item coverage 1/34 (union with kit 5/34) · set V-para dev-style (dev-style, development data) · judge jev · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-12 23:59 · caveat: packs are product value, not a bar lever - [F-094] Yield collapsed after the first week. 15 adopted improvements, all 10-04 → 10-08; ~70 later arms, 0 adopted points on the bounty metric · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI 159 arms · date 2026-10-14 · caveat: internal PT-BR page; counts from the arm index - [F-095] X2b no harm on the bounty-shaped kit. v0.1+simplewiki +0.012 (−0.019…+0.046); kit-only runs byte-identical · set v0.1 dev (dev, development data) · judge Haiku · n 57 · seeds UNKNOWN · CI n 57 · date 2026-10-14 · caveat: KIT-X2b closed at ceiling (R93) - [F-103] Run-to-run noise of identical configurations was Δ 0.031 (17/60 labels changed) 0.031 · set v0 dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI 60 items · date 2026-10-01 · caveat: motivated the MDE rule - [F-104] Run-to-run noise on the night of 2026-10-01 was Δ 0.031 with 17/60 labels changed. 0.031 · set v0.1 (status UNKNOWN) · judge Opus · n 60 · seeds UNKNOWN · CI n 60 · date 2026-10-01 · caveat: pre-dates the stats rules - [F-116] Bounty re-read: winning is a public confirmation, not a numeric benchmark. · date 2026-10-02 - [F-123] The public bench has 60 English questions; 57 are judged (3 venue-list items scored against OpenStreetMap instead). 60 / 57 · set v0.1, public dev (public dev, development data) · judge UNKNOWN · n 60 · seeds UNKNOWN · CI n 60 · date 2026-10-02 · caveat: 25 items quote/paraphrase a named public author's posts - [F-124] v0.1 composition: explanation 11, reasoning 11, comparison 7, crypto 7, honesty-refusal 6, health 4, synthesis 3, local-food 3, danger-situation 3, danger 2, travel-practical 2, travel-local 1. 12 categories · set v0.1 (status UNKNOWN) · judge UNKNOWN · n 60 · seeds UNKNOWN · CI n 60 · date 2026-10-02 - [F-125] The judge scores each answer 1-5 on three axes: correctness (a material wrong claim caps it at 2; an uncorrected false premise at 1), completeness, usefulness. 3 axes, 1-5 · set all sets (status UNKNOWN) · judge judge.v1 · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02 - [F-126] Above-decline ratio = (raw − D) / (1 − D) with D fixed at 0.306. formula (judge.v1 ratio) · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: different scale from floor-adjusted; does not use the floor - [F-127] CIs are a stratified (by category) two-level bootstrap: resample items within stratum, then runs within item; 2,000 resamples; seed 11; percentile 95 %. B 2000, seed 11 · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: does not include run-level noise beyond the runs available (see F-method-41) - [F-128] Calibration ladder on v0.1: Claude models with no web vs the Opus+web reference — Fable 1.048, Opus 1.029, Sonnet 0.950, Haiku 4.5 0.653 (raw). raw 1.048 / 1.029 / 0.950 / 0.653 (judge.v1 ratio) · set v0.1 dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI 1 run × 57 each; e.g. Opus 1.007-1.053 · date 2026-10-02 · caveat: the reference behaves like "frontier model memory", web adds little on these items - [F-129] v0.2's items and references were written by Claude-family subagents (two blind writers, 55 each); V-para by an Opus subagent. Claude-family · set v0.2, V-para (status UNKNOWN) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-02/03 · caveat: exact writer model id not recorded - [F-130] v0.2 writers converged: 36 same-topic pairs out of 55 between two Claude writers, then 7 of 37 again among rewriters. 36/55; 7/37 · set v0.2 (status UNKNOWN) · judge UNKNOWN · n 110 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: citation fix (l.44 is an instruction step) - [F-161] Fine-tune data under one strict faithfulness rubric: teacher 16/20, + sentence rejection 11/20, human CLAPnq+HAGRID 6/20 examples with an unsupported claim. 16 / 11 / 6 of 20 · set n/a (status UNKNOWN) · judge Opus reviewer · n UNKNOWN · seeds UNKNOWN · CI 20-example reviews · date 2026-10-07 · caveat: bar ≤ 2/20 unreachable even for human data - [F-164] Local Gemma 4 26B judge: same winner as Opus 49/57 and 50/57, κ 0.32 / 0.55, ratio +0.110 / +0.112 above Opus, 41-43 s/call. +0.11 (judge.v1 ratio) · set v0.1 dev (dev, development data) · judge Gemma vs Opus · n UNKNOWN · seeds UNKNOWN · CI 228 calls, 2 legs · date 2026-10-07 - [F-166] Position consistency (both orders agree) is 96.5-100 % for Opus and Haiku, 86-93 % for the local Gemma judge. 96.5-100 % · set v0.1 dev (dev, development data) · judge several · n 57 · seeds UNKNOWN · CI n 57 · date 2026-10-07 - [F-182] The reference answers score ≈ 4.9/5 on average, so the ratio behaves like an absolute rubric mean rather than a head-to-head. ≈ 4.9 (judge.v1 ratio) · set v0.2, V-para (status UNKNOWN) · judge Opus · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-10 · caveat: public citation fix: per-dimension reference means (4.63-5.00) are at l.88-91, not l.66-69 - [F-184] "20 hardest" slices inflated deltas: +0.091 slice vs +0.026 full (THINK4B); +0.089 vs +0.009 (THINK4B-V02). inflation ×3-10 · set dev (dev, development data) · judge Opus/Haiku · n 20 · seeds UNKNOWN · CI n 20 vs 51/110 · date 2026-10-10 - [F-185] 0 of 32 criteria frozen between 10-07 and 10-10 stated an expected Δ or MDE. 0/32 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10 · caveat: fixed by the Power block - [F-188] Formal raw rule (CI low > 0.50 on both sets) reading. met by T and H on dev data only (judge.v1 ratio) · set V-para + v0.2 dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-10 · caveat: H's v0.2 leg is 1 seed; desktop only - [F-190] Share of items with usefulness ≥ 3/5. K V-para 2.9 %, D 10.8 %, H 33.3 %; K v0.2 34.5 %, H 58.2 % · set V-para / v0.2 (status UNKNOWN) · judge Opus · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-10 - [F-199] A 20-item random slice has SE ≈ 0.04-0.05: it only screens for futility. SE 0.04-0.05 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n 20 · seeds UNKNOWN · CI n 20 · date 2026-10-11 - [F-200] Fail-closed judge: `--strict` exits before any call if a trace/reference is missing; `--floor-errored` scores a failed answer 1/1/1 instead of dropping it. flags · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-11 - [F-205] Bar test power at n = 110: required point estimate ≈ 0.53 without run noise, ≈ 0.545 with one-seed run noise; a true 0.53 passes only ≈ 29-50 % of the time. 0.53 / 0.545 · set v0.3 plan (status UNKNOWN) · judge UNKNOWN · n 110 · seeds UNKNOWN · CI n 110 · date 2026-10-12 · caveat: t-based, not bootstrap - [F-209] V-para: 58 items (51 judged + 7 venue), 19 full generation runs and 12 slices of 20. 58 / 19 / 12 · set V-para dev-style (dev-style, development data) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-13 - [F-211] The planned 40-pair blind human label check (seed 7, stratified) has not been done. 0/40 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 - [F-217] The v0.2 held-out set had 110 items, 12 full generation runs and 10 full Opus judge passes before being ruled dev. 110 / 12 / 10 · set v0.2 burnt (burnt → dev, development data) · judge Opus · n 110 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-14 - [F-218] Of 9 private blind sets, only one (a 32-item Spanish set) is still clean. 1/9 clean · set all blind sets (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: Spanish line parked; not usable for the English claim - [F-222] X1 (append neighbour chunks of retrieved article, 0 LLM) +0.033 (+0.016…+0.050) Opus; fact present 35 → 72 · set 180 self-tests dev (dev, development data) · judge Opus (beyond frozen plan) · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-14 · caveat: frozen label "not decidable at MDE 0.03" | verdict capped "not decidable": power line missed by 3 items - [F-227] Critique: most traced numbers re-derive. 15/17 numbers, 16/17 criterion hashes match · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 - [F-228] Judge-auth outage. judge passes impossible ≈ 06:4x → 09:0x on 10-16 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: no pass lost - [F-231] The exposure ledger holds 126 dated rows (55 v0.2-only, 33 V-para-only, 19 both). 126 dated rows (55 v0.2-only, 33 V-para-only, 19 both) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-17 · caveat: recount of rows starting '| 10-' in the Ledger section gives 126, not 128; the 55/33/19 split is confirmed - [F-232] The Claude judge is the stricter family by ≈ 0.10 raw on both sets. offset +0.08…+0.11 raw (+0.08/+0.115 V-para) (judge.v1 ratio) · set V-para, v0.2 (status UNKNOWN) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-17 · caveat: ranking unchanged - [F-235] Combining the two orders: same winner keeps it, disagreement becomes a tie; rubric scores are averaged. rule · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-236] Raw ratio = Σ S / Σ R, where S and R are the app's and reference's mean rubric (mean of 3 axes, both orders averaged). formula (judge.v1 ratio) · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-237] Floor-adjusted ratio = Σ (S − 1) / Σ (R − 1): quality above the rubric minimum of 1; 0.50 here means "half as good as the reference". formula (judge.v1 ratio) · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-239] Paired differences between two systems use the same stratified two-level bootstrap on the three scales. B 2000, seed 11 · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-241] Every judgment row records judgeModel, cliVersion, judgeVersion and the three input shas. 6 provenance fields · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-242] Judge versions are tagged and never pooled: judge.v1 (Opus), judge.v1-low (cheap effort), judge.v1-med. tags (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: citation fix: the version tag is built at l.33 - [F-243] Everything after the judge (traces → metrics) is deterministic given the judgment rows. deterministic · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-263] D = 0.306 = mean judge rubric of 79 declined answers across 7 runs (≈ 1.50/5) divided by the mean reference rubric; v0.2-weighted 0.307. 0.306 · set v0.1 night runs, dev (dev, development data) · judge Opus · n 79 · seeds UNKNOWN · CI n 79 declines, 7 runs · date 2026-10-01 - [F-268] On the 4B, raw 0.512 = 0.387 floor-removed = 0.297 above always-decline. 0.512 / 0.387 / 0.297 (judge.v1 ratio) · set v0.1 dev (dev, development data) · judge Opus · n UNKNOWN · seeds 2 · CI 2 seeds; floor-removed CI 0.319–0.448 · date 2026-10-02 · caveat: three scales for one run; v0.1 dev, 2026-10-02; the near-identical 0.512 on v0.2 (F-002) is a different set. - [F-270] Cross-family rule in the plan: if a second-family judge differs by > 0.05 raw, the lower of the two is the headline; report κ between judges. 0.05 (judge.v1 ratio) · set v0.2/v0.3 (status UNKNOWN) · judge UNKNOWN · n 110 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-02 - [F-284] On correctness alone H is above 0.50 floor-adj; the gap is in completeness and usefulness. V-para correctness floor-adj 0.660 (0.608–0.712) vs completeness 0.301 / usefulness 0.341 · set V-para dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 2 · CI 2 seeds · date 2026-10-10 · caveat: "correct but thin" also describes an always-decline system on those dims; not a bar reading - [F-311] Lexical health detector tuned on dev failed a fresh blind set: health route 31/72 (bar ≥ 54) 10 → 31/72 · set health-fresh1 (blind then; dev now) (dev, development data) · judge scripted · n UNKNOWN · seeds 1 · CI 1 seed · date 2026-10-03 - [F-312] Health stack L2: unsafe answers 7 → 0 (4B) and 10 → 0 (1.5B) with no quality cost. ΔQ +0.009 (−0.009…+0.030) · set v0.1 dev (dev, development data) · judge Opus + safety rules v0.8.2 · n UNKNOWN · seeds 2 · CI 2 seeds · date 2026-10-03 · caveat: cards were unvetted drafts at the time - [F-313] The independent Opus reviewer changed 25 of 31 first-aid cards, rejected 0. 6 approved / 25 changed · set n/a (status UNKNOWN) · judge Opus reviewer · n UNKNOWN · seeds UNKNOWN · CI 31 cards · date 2026-10-03 · caveat: "AI-reviewed, not clinically reviewed" - [F-314] Oracle ladder: closed-book 0.487 · engine 0.507 · question-chosen passages 0.567 · gold passages 0.606 · 1500-token gold 0.626 · Haiku no-web 0.653. see value · set v0.1 EN dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI 57 items · date 2026-10-03 · caveat: gold passages chosen with knowledge of the reference → upper bound - [F-317] 4B loss anatomy: gap to Haiku +0.121 net; cut 2's gain over old main (+0.026) came entirely from non-model paths. +0.121 · set v0.1 EN dev (dev, development data) · judge existing verdicts · n UNKNOWN · seeds UNKNOWN · CI 57 items · date 2026-10-04 · caveat: biggest buckets "thin but correct" and "knowledge" had no lever - [F-318] Cut 3 over cut 2 on v0.1. +0.032 CI90 [+0.010, +0.062]; 0.566 vs 0.534 · set v0.1 dev (dev, development data) · judge Opus judge.v1-med · n 57 · seeds UNKNOWN · CI n 57 · date 2026-10-04 · caveat: last 4B-side adopted quality gain | rests on 4 items; did not transfer to held-out (F-chron-early-62) - [F-319] Cut 2's gain over old main came from non-model paths. +0.028 cards + calculator; model-side ≈ 0 · set v0.1 dev (dev, development data) · judge not stated at the cited line (neighbouring buckets are in Haiku-rubric terms) · n 57 · seeds UNKNOWN · CI n 57 · date 2026-10-04 · caveat: citation fix: the +0.028 non-model-path finding is l.7 - [F-341] An app that declines every question scores raw ≈ 0.31, i.e. ≈ 61 % of the 0.50 bar for free. 0.306 / 61 % (judge.v1 ratio) · set v0.1 (status UNKNOWN) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-06 · caveat: citation fix: l.13 is the cut 6 v0.2 row - [F-342] Cut 6 vs cut 3 on v0.2 is no change. Δ −0.0009 · set v0.2 held-out at time (held-out at run time) · judge Opus · n 110 · seeds UNKNOWN · CI −0.0095…+0.0077; n 110 · date 2026-10-06 · caveat: citation fix (l.48 is a separator). 323 of 334 verdicts were copied from cut 3 for byte-identical answers, so the narrow Δ reflects reuse, not an independent re-judging - [F-345] DETMAX fixes chit-chat prefixes before an emergency. lost routes 7/240 → 0/240; split-turn forms lose 0/60; +1 route in 192 dev chats (a real flood emergency) · set dev probes (dev, development data) · judge none · n 240 · seeds UNKNOWN · CI n 240 / 60 / 192 · date 2026-10-06 · caveat: citation fix (l.46 blank) - [F-346] FACTS asset size. 4,233 entities, 760,970 B · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-06 · caveat: citation fix (l.6 is the sha line) - [F-358] Cut 7 build identity. commit 6d0b93b1, tree 7c96e59e…, patch vs cut 6 sha 0eee6a0b…; 1,346/1,346 unit tests · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-07 · caveat: local commit, never pushed - [F-359] FACTS (Wikidata table) fires zero times on the held-out sets, so cut 7 = cut 6 there. 0/110 v0.2, 0/58 V-para; 8/72 on v0.1 · set v0.2 (burnt→dev), V-para (burnt → dev, development data) · judge none · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-07 - [F-363] Thinking budget 3072 added nothing: the model stops by itself at ~1.3k tokens. Δ +0.010 (−0.014…+0.041) · set V-para dev-style (dev-style, development data) · judge Haiku L1 · n UNKNOWN · seeds 1 · CI 1 seed · date 2026-10-08 - [F-365] References are ≈ 280 words (p10/p50/p90 = 251/280/318); the 4B app's answers median ≈ 51-53 words. 280 vs ~52 words · set V-para / v0.2 (status UNKNOWN) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-10 · caveat: citation fix: 53 words (cut 6 v0.2) at PIPELINE-AUDIT l.17, 51 words (v0.1) at RND-REVIEW l.50; 251/280/318 at BAR-TABLE l.95 - [F-371] Resume safety: a cached judgment row is reused only if traces, questions and references sha256, judge version and model all match; else exit 7. exit 7 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 - [F-375] Health questions on the default tier never reach the 4B: they get a fixed card or a verbatim source excerpt (0 model tokens). modelId grounding-guard / extractive · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-15 · caveat: citation fix; design statement; AI-reviewed, not clinically reviewed - [F-378] HEALTHGAP: 15 of 64 health-shaped probes reached the model because the detector scored them 0.00–0.16 (dose, poison, CO reading, survival time). 15/64 · set agent-written dev probes (dev, development data) · judge none · n 64 · seeds UNKNOWN · CI n 64 probes · date 2026-10-15 · caveat: citation fix (l.156-158 are open questions) - [F-390] Lite tier safety. health routing identical to 4B 124/124; model answers alone 3/64 health probes; unsafe 1 vs 1 · set agent-written probes + v0.1 (status UNKNOWN) · judge jev + rules · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: claim kit only, one seed - [F-433] Asking the 4B for longer answers (persona P2) lowered quality: correct items 39 % → 14 %, confident-wrong 10 → 20. Δ −0.044 · set v0.1 dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI (−0.076…−0.014), MDE 0.029 · date 2026-10-02 · caveat: judge length slope negative (−0.34 per log-word) - [F-441] A 4B one-token emergency gate generalized on two blind sets; lexical did not. 90/90, HN 6/60; 71/72, HN 2/48 (lexical 23/90, 32/72) · set health-fresh2/1 (blind then; dev now) (dev, development data) · judge scripted · n UNKNOWN · seeds UNKNOWN · CI detection only · date 2026-10-03 - [F-443] Retrieval v2 put the gold sentence in context 7 → 18/56 times but lowered quality. Δ −0.021 · set v0.1 EN dev (dev, development data) · judge Opus · n UNKNOWN · seeds 2 · CI CI90 −0.054…+0.009, 2 seeds · date 2026-10-03 · caveat: retrieval 3.6× slower; 3 items carry the loss - [F-450] Bigger closed-book models do not help the 4B tier. Qwen3.5-4B 0.437, Gemma E4B 0.495, Qwen3.5-9B 0.488, Ministral 8B 0.457 vs 0.487 · set v0.1 dev (dev, development data) · judge Opus · n 57 · seeds 1 · CI n 57; 1 seed · date 2026-10-05/06 · caveat: not detected at MDE 0.055 - [F-453] Retrieval failure scorecard on v0.1 gold items. coverage bucket 49 %; recall@5 0.333; gold in prompt 24/78 item-runs · set v0.1 dev (dev, development data) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI 39 gold items × 2 runs · date 2026-10-06 - [F-455] Deep model reads gold passages like the 4B. R450 0.634; paired vs 4B +0.029 (−0.004…+0.063) · set v0.1 dev (dev, development data) · judge Opus · n 57 · seeds 1 · CI 1 seed; n 57 · date 2026-10-06 · caveat: deep gain = knowledge, not reading - [F-459] Deep tier vs 4B paired gain on V-para. +0.068 floor-adj · set V-para dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds UNKNOWN · CI (two-level paired gap file) · date 2026-10-08 - [F-460] H vs the 4B on v0.2. +0.089 raw; +0.111 floor-adj (judge.v1 ratio) · set v0.2 dev (dev, development data) · judge Opus · n 110 (inferred from set) · seeds 1 · CI raw +0.054…+0.126; floor +0.066…+0.159; 1 seed · date 2026-10-09 - [F-463] The 4B's two seeds on V-para give identical ratios (Δ 0.000, −0.023…+0.023). 0.000 · set V-para (status UNKNOWN) · judge Opus · n 51 · seeds UNKNOWN · CI n 51 · date 2026-10-10 - [F-464] The 4B is seed-stable on V-para (identical seeds) per-item r 0.81; Δ 0.000 · set V-para (status UNKNOWN) · judge Opus · n 51 (inferred from set) · seeds 2 · CI 2 seeds · date 2026-10-10 - [F-465] Thinking twin of the claim 4B (Qwen3-4B-Thinking-2507 @512) on V-para. raw 0.439; floor-adj 0.293; Δ +0.026 vs K (judge.v1 ratio) · set V-para dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 1 · CI Δ CI −0.013…+0.068; 1 seed · date 2026-10-10 · caveat: ×3.7 time (23.6 s); not detected at MDE 0.06 - [F-466] 4B correctness-only floor-adj on V-para / v0.2. 0.427 (0.371–0.482) / 0.595 (0.544–0.646) · set V-para / v0.2 dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 2 · CI 2 seeds · date 2026-10-10 - [F-467] Shortening references to ≤ 80 words lifts H floor-adj 0.412 → 0.514 and K 0.261 → 0.322. +0.102 (+0.068…+0.144) H; +0.060 K · set V-para dev, references altered (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 1 · CI 1 seed (s42) · date 2026-10-10 · caveat: ≈ 17 % of H's gap is reference length, ≈ 83 % substance; not a bar reading - [F-468] Reference answers are long. p10/p50/p90 = 251/280/318 words · set V-para (status UNKNOWN) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-10 - [F-470] Numbers and dates dominate the wrong-fact loss; declining instead of a wrong answer would lower the score (RATIONALEPASS) numeric + date 0.51 (V-para) / 0.70 (v0.2) of wrong loss; decline oracle −0.064 / −0.040 · set V-para / v0.2 (status UNKNOWN) · judge jev over Opus rationales · n UNKNOWN · seeds UNKNOWN · CI 0/16, 0/17 items better · date 2026-10-12 14:17 - [F-476] 4B generation on desktop CPU is byte-deterministic. 20/20 identical across 5 pairings · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 · caveat: citation fix (l.16 is the stack bullet) - [F-487] Registry size. 98 rows private (R1–R98); 76 rows public · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: public lags R77–R98 - [F-488] Cross-family judge on v0.2: 4B raw 0.617 (0.567-0.666) / floor-adj 0.517 vs Claude 0.512 / 0.388; deep raw 0.714 (0.666-0.759) / floor-adj 0.639 (0.579-0.695) vs 0.608 / 0.508. ≈ +0.10 raw / +0.13 floor (judge.v1 ratio) · set v0.2, burnt → dev (burnt → dev, development data) · judge cross-family vs Opus (post-hoc, different judge family) · n 110 · seeds 2 · CI 660 calls, n 110; 4B 2 seeds, deep 1 seed · date 2026-10-17 - [F-490] Paired deep − 4B on v0.2 is identical under both judges. +0.097 (+0.053…+0.142) vs Claude +0.096 (+0.062…+0.131) · set v0.2 dev (dev, development data) · judge both · n 110 (inferred from set) · seeds UNKNOWN · CI (+0.053…+0.142) / (+0.062…+0.131) · date 2026-10-17 · caveat: Spearman 0.90–0.97 | on V-para +0.171 vs +0.139 - [F-578] Deep 35B reading the same gold passages as the 4B scored only slightly higher: the reader converts like the 4B; engine-level +0.06 is knowledge. 0.634 vs 0.606; Δ +0.029 · set v0.1 EN dev (dev, development data) · judge Opus · n UNKNOWN · seeds 1 · CI (−0.004…+0.063), 57 items, 1 seed · date 2026-10-06 · caveat: "not detected at MDE 0.055" - [F-579] A completeness instruction made the 35B longer but less correct. correctness 3.89 → 3.10; CW 7 → 15; ratio Δ −0.014 (judge.v1 ratio) · set v0.1 EN dev (dev, development data) · judge Opus · n UNKNOWN · seeds 1 · CI (−0.055…+0.024), 1 seed · date 2026-10-06 - [F-583] The Gemma judge preserved the ranking: deep35 − 4B +0.026 vs Opus +0.029. +0.026 vs +0.029 · set v0.1 dev (dev, development data) · judge both · n 57 · seeds UNKNOWN · CI n 57 · date 2026-10-07 · caveat: neither CI excludes 0 - [F-585] Fast-tier constants in cut 7. retrieve k 6, context 1,200 tokens (lookup 3/450; other 4/700 quick budget), fast thinking budget 256 (unused by the instruct 4B) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-07 · caveat: instant final confidence 0.75 (context.ts:139) - [F-593] Budget curve (V-para floor-adj): 0 → 256 → 512 → 1024 → 3072. 0.329 → 0.387 → 0.436 → 0.422 → no further gain · set V-para dev (dev, development data) · judge Opus (3072 Haiku) · n 51 (inferred from set) · seeds 2 · CI 2 seeds each (3072: 1 seed Haiku, +0.010) · date 2026-10-08/09 · caveat: knee at 512; Qwen stops thinking ≈ 1.3k tokens - [F-605] Haiku L1 stopped the v0.2 thinking arm at +0.027 (vs +0.03 clause); Opus later measured +0.045 (+0.022…+0.070) +0.027 Haiku / +0.045 Opus · set v0.2 dev (dev, development data) · judge Haiku L1 / Opus · n 110 · seeds Haiku 1; Opus 2 · CI Haiku s42: +0.027 (+0.002…+0.054), n 110; Opus two seeds two-level: +0.045 (+0.022…+0.070); Opus same-seed s42 comparison: +0.048 · date 2026-10-09 · caveat: +0.045 is the two-seed figure (not '1 seed'); the near-miss rule text quotes the same-seed +0.048. Both are correct in context - [F-606] Budget 256 vs 1024 / vs 512. −0.028 / −0.039 raw (judge.v1 ratio) · set V-para dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 2 · CI −0.056…−0.003 / −0.066…−0.015; 2 seeds · date 2026-10-09 · caveat: keeps ≈ 63 % of the thinking gain at 40 % of the time - [F-607] Thinking budget hits: at 1024, 58/91 generating v0.2 items and 36/51 V-para; at 512, 83/110 v0.2 and 46/51 V-para. 1024: 58/91 generating v0.2 items (s42) and 36/51 V-para (s42); 512: 83/110 v0.2 (s42) and 46/51 (s42) / 42/51 (s43) V-para · set v0.2 / V-para dev (dev, development data) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-09 · caveat: the 36/51 figure was uncited (it is DEEPTHINK.md l.16); counts are per seed - [F-608] Gemma + thinking vs Qwen + thinking. −0.028 raw (judge.v1 ratio) · set V-para dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 2 · CI −0.057…+0.003; 2 seeds · date 2026-10-09 · caveat: parity-faster clause failed - [F-618] Per-item seed agreement: deep thinking tier r 0.5-0.7 on V-para vs 0.81 for the 4B; on v0.2 the deep tier r 0.90. r · set V-para / v0.2 dev (dev, development data) · judge Opus · n 51 · seeds UNKNOWN · CI n 51 / 110 · date 2026-10-10 - [F-622] Deep-tier constants. retrieve k 6, context 400 tokens, max 200 answer tokens; thinking budget 1024 in code, 512 recommended · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10 - [F-630] Where the deep tier loses points (LOSSMAP) absent knowledge 0.59 (V-para) / 0.55 (v0.2) of loss; wrong fact 0.40 / 0.21 · set V-para dev-style + v0.2 dev (dev-style, development data) · judge Opus rationales classified by jev (a calibrated classifier) · n UNKNOWN · seeds 1 · CI 51 / 110 items; 1 seed · date 2026-10-11 12:10 · caveat: label model unvalidated against humans (CRITIQUE C3) - [F-641] A teacher-model probe ran 10.1 mini-hours without a smoke test; 45/120 answers were empty because thinking and answer shared one token budget. 10.1 h; 45/120 empty; 6/120 passing · set dev (dev, development data) · judge jev check · n 120 · seeds UNKNOWN · CI n 120 · date 2026-10-14 · caveat: a 10-item smoke (≈ 54 min) would have shown it - [F-649] Blinding strips think blocks, trailing "Sources:" lists, URLs, markdown links, [n] markers and bold before judging. 1 function · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: stripping sources also removes web provenance signals from the reference - [F-650] The deep-tier model is Qwen3.6-35B-A3B UD-Q2_K_XL. 12,290,628,576 B, Apache-2.0 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10 · caveat: desktop only so far - [F-682] Bounty hard limits: Android/GrapheneOS, ≤ 12 GB RAM, ≤ 50 GB assets, fully offline, no Play Services, public repo. · set n/a · judge n/a · date 2026-10-02 · caveat: requirements text, not BOAR's - [F-684] The desktop proxy matched a real iPhone 13 run. 0.394 vs 0.395; Δ −0.001 · set v0.1 dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI (0.353–0.441), MDE 0.038, 2 reps · date 2026-10-02 · caveat: a second Claude-family judge agreed (same winner 99 %, κ 0.894) QUEUE:249 - [F-686] On an 8 GB Android the 4B was killed by the OS while loading; ARM weight repack copied ~2.3 GiB into anonymous memory. RSS 4.7–5.0 GiB at kill · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI 2 reproductions · date 2026-10-02 · caveat: debug build; fix tested on desktop only (anon 3.25 → 0.88 GB) - [F-691] The 4B weight repack on ARM duplicates ~2.3 GiB in anonymous memory. Probe A: anon 3.25 → 0.88–0.91 GB, max RSS 5.1–5.4 → 3.41 GB with no-repack · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: macOS desktop probe - [F-701] Cut lineage. cut 1 → 1b → 2 → 3 (VG v3 + AR + snakebite) → 4 (KVFIX + INSTANT-FRAG + INSTANT-SUBJ) → 5 (KV no-source prefix + gate skip) → 6 (DETMAX) → 7 (FACTS r6) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-03..07 · caveat: base = public upstream commit 0cfd4a77 - [F-710] Most phone latency came from a KV-session save bug in OUR patch (desktop shim no-op hid it); ~55–65 s re-prefill per answer. 55–65 s · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-05 · caveat: not an upstream bug - [F-715] KV-cache save bug cost on the A15. gate prefix 35–41 s + answer prefix 20–28 s re-prefilled per question at ~12 tok/s; fix est. −55 to −65 s · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-05 · caveat: our own regression, not upstream (QUEUE:649) - [F-716] q8_0 KV cache is non-inferior on quality. Δ −0.0077 (−0.0232…+0.0062); reader KV 576 → 306 MiB · set v0.1 dev (dev, development data) · judge Opus · n 60 · seeds 2 · CI n 60 × 2 seeds · date 2026-10-05 · caveat: desktop slower; device-day candidate only | device candidate - [F-740] bmoe desktop speed and memory. prefill 25.7 s median, decode 8.6 tok/s, session peak 11.7 GiB on macOS (Linux anon 4.0–4.4 GiB) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-07 · caveat: accounting differs by OS - [F-750] Gemma 4 26B-A4B + thinking (G) on V-para. raw 0.514; floor-adj 0.388 (judge.v1 ratio) · set V-para dev (dev, development data) · judge Opus · n 51 (inferred from set) · seeds 2 · CI raw 0.487–0.540; floor 0.354–0.421; 2 seeds · date 2026-10-09 · caveat: 27 % faster than T, RSS ≈ 7.7 GB - [F-751] bmoe v0.28.0 has no thinking-budget flag; a device needs an app-side cap. · date 2026-10-09 · caveat: unbounded thinking ≈ 1.3k tokens, ≈ 2× time - [F-756] The same 35B model file was deleted twice under queued experiments (once by an app-packaging disk cleanup, once by a model swap), costing two 12.3 GB re-downloads. 2 × 12.3 GB re-downloads · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10, 2026-10-12 · caveat: → rule 22 | Verification note: The same 35B Q2_K_XL file was deleted twice: 10-10 22:42 by the APK track's disk cleanup, 10-12 08:21 by the coordinator's QUANTCEIL model swap (not housekeeping both times). - [F-766] Deep + think 512 on a Pixel 8 Pro CPU is projected at 1.9–2.8 min per answer. 1.9–2.8 min · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI from published third-party rates · date 2026-10-12 · caveat: not our measurement; deep tier never ran on a phone - [F-771] Phone speed (cut 7, POCO X6 Pro, MT6897, 10.7 GiB) fast tier p50 30.7 s / p90 48.4 s; TTFT p50 18.5 s; decode 5.5 tok/s; all 72 p50 24.7 s · set v0.1 public (public dev, development data) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI 72/72 answered · date 2026-10-14 · caveat: airplane mode, no INTERNET permission - [F-772] Phone memory (cut 7) PSS peak 5.88 GB at load, then p50 3.64 / max 4.23 GB · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: thermal status 0 over 29 min - [F-773] Phone vs desktop speed ratio. total p50 28.3 s vs 7.5 s (×3.4); decode 5.6 vs 22.6 tok/s (judge.v1 ratio) · set v0.1 (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI 60 ids · date 2026-10-14 · caveat: desktop = MacBook M1 Pro - [F-775] Cut 7 on the POCO X6 Pro (MT6897, 10.7 GiB, Android 16, airplane, no INTERNET permission). 72/72 success, 0 timeouts, logcat clean · set v0.1 public + safety (public dev, development data) · judge UNKNOWN · n 72 · seeds UNKNOWN · CI n 72 + warm-up · date 2026-10-14 - [F-776] X6 fast-tier latency. total p50 30.7 s, p90 48.4 s, max 53.0 s; TTFT p50 18.5 s; decode p50 5.5 tok/s (4.6–7.4); output p50 64 tokens · set v0.1 (status UNKNOWN) · judge UNKNOWN · n 51 · seeds UNKNOWN · CI n 51 fast · date 2026-10-14 - [F-777] X6 instant tier. p50 0.1 s, p90 11.2 s; 21 instant (guard 12, extractive 4, places 3, calculator 2) · set v0.1 (status UNKNOWN) · judge UNKNOWN · n 21 · seeds UNKNOWN · CI n 21 · date 2026-10-14 - [F-778] X6 memory. PSS peak 5.88 GB during model load, then p50 3.64 / p90 3.80 / max 4.23 GB · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 - [F-779] X6 thermal. battery max 41.7 °C, thermal_status 0 throughout · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: charging during run - [F-780] X6 parity with desktop. tier 60/60, instant 16/16 byte-identical, kept titles 59/60, 4B text identical 3/44 (sampler differences); phone 3.4× slower (decode 22.6 vs 5.6 tok/s) · set v0.1 (status UNKNOWN) · judge UNKNOWN · n 60 · seeds UNKNOWN · CI n 60 · date 2026-10-14 - [F-781] X6 judged quality: device = desktop. raw 0.529 (0.481–0.575) vs 0.529 (0.484–0.575); floor-adj 0.403 vs 0.402; paired −0.000 (−0.026…+0.028); win/tie/loss 2/1/54 vs 1/2/54 (judge.v1 ratio) · set v0.1 public (dev) (public dev, development data) · judge Haiku L1 · n 57, · seeds UNKNOWN · CI n 57, 1 run · date 2026-10-14 · caveat: Haiku vs A15's Opus: levels not comparable - [F-793] LES (expert streaming) on a phone is projected at ≈ 4 tok/s (UFS 3.1) to ≈ 6 tok/s (UFS 4.0) including compute; analytic, unmeasured. with compute ≈ 4 tok/s (UFS 3.1) and ≈ 6 tok/s (UFS 4.0); I/O-only upper bounds 4.8 / 8.0 tok/s · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI analytic · date 2026-10-16 · caveat: '6-8' mixes the compute-included estimate with the I/O-only bound; analytic, unmeasured - [F-795] Phone flash bound for expert streaming. 320 expert reads/token, 314 MiB/token, 0 cache hits → ≈ 4 tok/s (UFS 3.1) to ≈ 6–8 tok/s (UFS 4.0) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI analytic · date 2026-10-16 · caveat: not measured on a phone - [F-823] Research agents work in fixed lanes, write one "technique card" per idea, never run experiments or touch the product; only the coordinator queues runs — why: Avoid duplicate work; separate proposing from measuring — outcome: the retrieval research agent 19 cards, the models/latency research agent 22, the evaluation/safety research agent 23; runner agent executed. · date 2026-10-01 - [F-843] 65/65 criterion sha256 files verified, and the freeze commit precedes the result commit. 65/65 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10 · caveat: ledger audit 10-13 later found 41 criteria without a sidecar sha (21 backing a decision) and one criterion edited after freezing - [F-846] The 10-10 process review. 10 Opus agents, 76 findings; double-check: 32 claims → 10 verified, 20 partly, 2 false · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10 - [F-873] Closed arms by the stage at which they died. 0-call 54 · Haiku L1 only 10 · L1 → Opus 12 · Opus direct 25 (of 101) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 · caveat: Opus-direct mostly before Haiku L1 existed - [F-893] Loop totals 2026-10-11 12:10 → 10-14 19:09: ≈ 12.0k Claude calls, of which ≈ 10.5k judge calls. 133 ticks (28 idle); ≈ 12.0k Claude calls = Opus 4,851 judge + Haiku 7,151 (5,660 judge + 1,491 writer/generation), i.e. ≈ 10.5k judge calls; mini 44.4 h + MacBook 18.1 h generation; 1 LEVER (dev) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: ≈ 1,440 Opus calls avoidable (Haiku picked the same winner) | Verification note: Calling all 12.0k "judge calls" overstates judging by ≈ 1.5k; also fix in the §1 summary. - [F-895] Long-session latency grows with history. ≈ 22 s at turns 14–15 vs ≈ 8 s standalone (desktop); prompt +74 tokens/turn; 4,096-token context crossed at ≈ turn 50 · set dev (dev, development data) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI 3 sessions · date 10-14 · caveat: adapter passes all turns; the app keeps 6 messages + summary from turn 8 (turns 5–7 gap) - [F-899] Rule changes logged. 29 numbered rules + 27a in 12 days (10-03 → 10-14) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: public copy omits caps and personal quotes - [F-900] Unattended loop activity 10-11 → 10-14. 133 ticks; 28 idle (21 %); 34 event-driven; 10 META; 3 restarts · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: another doc counts 37/121 idle (different window) - [F-903] Machine generation time and idle time. mini 44.4 h, MacBook 18.1 h; both idle 25.6 of 55.5 window-hours (09-24 h) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI estimate · date 2026-10-14 · caveat: 4 MacBook intervals estimated - [F-915] How arms died. 101 closed arms: 54 at 0 judge calls, 10 at Haiku L1, 12 at L1→Opus, 25 Opus directly · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: citation fix - [F-916] Multi-turn sessions do not degrade answers over 5 turns. turns 2–5 ratio in-session/standalone 1.397 (1.187–1.673); drift +0.537 (+0.208…+1.155) (judge.v1 ratio) · set agent-written dev (dev, development data) · judge Haiku L1 · n UNKNOWN · seeds 1 · CI 10 sessions × 5 turns; 1 seed · date 2026-10-14 · caveat: citation fix (l.3 is the health caveat). Reference = BOAR's own standalone answer, not frontier; drift is a difference of ratios - [F-932] Closed-family registry size. public 76 rows (61 closed, 13 not detected, 2 measurement); private 98 rows by 10-16 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-15/16 · caveat: public snapshot frozen 10-15 | Verification note: 76 = 61 + 13 + 2 is at REVIEW-RESEARCH-PAGE:5, not :33. - [F-953] Lane agents wrote technique cards. the retrieval research agent 19, the models/latency research agent 22, the evaluation/safety research agent 23 cards · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date by 2026-10-17 · caveat: count of files, not of tested ideas - [F-959] One full v0.1 judge pass is 114 judge calls (57 items × 2 orders). 114 calls · set v0.1 (status UNKNOWN) · judge Opus judge.v1 · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-990] v0.2-new items sit mostly outside the claim kit's knowledge packs: 72 % not-in-kit (kit A). 79/110 · set v0.2 (status UNKNOWN) · judge UNKNOWN · n 110 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-02 - [F-992] Perfect Wikipedia retrieval buys at most ~+0.005–0.017; most oracle gain came from web-only facts. web-only +0.027, wiki-only +0.005 · set v0.1 EN dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-03 - [F-993] Big pack (top-100k Wikipedia) on the 4B: no gain, retrieval 3.3× slower. Δ −0.005 / −0.016 · set v0.1 EN dev (dev, development data) · judge Opus-med · n UNKNOWN · seeds 1 · CI 1 seed · date 2026-10-03 · caveat: one seed: under-powered (audit) | Verification note: Public CLOSED.md:20 (R5) is the 10-01 #6 v1 gate, not exp-33 (10-03). No dedicated public row for exp-33. One seed, under-powered. - [F-995] The original kit size. 3.03 GB including the 2.5 GB model; packs ≈ 0.39 GB · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-05 - [F-996] Haiku 5.5-low vs Opus on 57 v0.1 items: same winner 54/57, r 0.922, κ 0.651, ratio −0.069 lower, 3.52 s/call, 0 failures. 54/57 (judge.v1 ratio) · set v0.1 dev (dev, development data) · judge Haiku vs Opus · n 57, · seeds UNKNOWN · CI n 57, 228 calls · date 2026-10-07 · caveat: 'same winner 54/57' carries almost no information because the reference wins ≈ 100 % of items (CRITIQUE-2026-10-16.md:32); quote the informative statistics instead: r 0.922, κ 0.651, level −0.069, Haiku Δ ≈ 0.70-0.85× the Opus Δ on pack arms - [F-999] Adding the top-100k Wikipedia pack to the deep tier crowded out curated chunks. Δ −0.034 (−0.082…+0.003); 26/28 slots taken · set V-para dev-style (dev-style, development data) · judge Haiku L1 · n UNKNOWN · seeds UNKNOWN · CI slice 20 · date 2026-10-10 · caveat: corpus size closed for deep tier too - [F-1000] Retrieval still matters for the deep tier (removing packs hurts) −0.045 (−0.083…−0.011) · set V-para slice dev (dev, development data) · judge Haiku N=4 · n 20 (slice) · seeds 1 · CI slice 20; 1 seed · date 2026-10-10 · caveat: slice power low - [F-1007] Pipeline metric for the first loop cycle. 0 V-para floor-adj points adopted per 1,804 judge calls (final tally of the 10-11 → 10-12 cycle); 1,180 was an interim count in BACKLOG · set V-para (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-12 · caveat: Verification note: Cross-file: chron-late F-18 uses 1,804. - [F-1008] Wikipedia-anywhere gate for V-para absent-knowledge items. 9/34 verified (MARGINAL band 7–13) · set V-para dev (dev, development data) · judge jev + quote check · n 51 (inferred from set) · seeds UNKNOWN · CI 34 items · date 2026-10-12/13 · caveat: 25/34 unverified, not shown absent - [F-1025] Pack lift per pack tracks the keep step. simplewiki +0.102, ethereum-SE +0.074, bitcoin-SE +0.054; first aid +0.004; factbook −0.018 · set 180 self-tests dev (dev, development data) · judge Haiku · n 180 (inferred from set) · seeds UNKNOWN · CI 20 per pack · date 2026-10-13 · caveat: citation fix; descriptive, 20 items per pack, Haiku - [F-1026] Top-1 pack routing fails. accuracy 0.25 (vote) / 0.58–0.61 (4B zero-shot) vs 0.85 bar · set 180 self-tests (dev, development data) · judge none · n 180 (inferred from set) · seeds UNKNOWN · CI 0-call / local · date 2026-10-13 · caveat: break-even ≈ 0.68 | break-even ≈ 0.68 accuracy - [F-1027] Fact chunk reaches the 4B context today. kept 43/180; raw top-6 55/180 (judge.v1 ratio) · set 180 self-tests (status UNKNOWN) · judge UNKNOWN · n 180 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-13 - [F-1028] Published packs on the author's public Hugging Face dataset (boar-packs). 11 packs: qa/ 7, topics/ 4 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 · caveat: the dataset URL carries the author's own handle; whether to print it is the author's decision - [F-1030] Pack licences. Stack Exchange CC BY-SA 2.5/3.0/4.0 per post (archive.org dump 2024-04-02); simplewiki CC BY-SA 4.0 (+GFDL); factbook CC0; MedlinePlus public domain (credit line); firstaid CC BY-SA + US PD · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-13 · caveat: SE dump is 2.5 years stale - [F-1063] X2b conversion where the fact chunk is newly present. +0.177 (+0.098…+0.263) · set 180 self-tests dev (dev, development data) · judge Opus · n 30 · seeds UNKNOWN · CI n 30 · date 2026-10-14 · caveat: appending non-fact chunks ≈ 0 cost (+0.008) | — - [F-1064] X2b reach and cost. fact present 102/180 (S 35); +≈ 420 prompt tokens p50; total p50 13.4 s vs 8.3 s; sidecars ≈ +60 % pack size · set 180 (status UNKNOWN) · judge UNKNOWN · n 180 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-14 - [F-1072] Detector v3b dev numbers at threshold 0.2. curated recall 179/180, strict false fires 15/100, ES 54/54, 0 critical misses; realistic recall 240/246, false fires 3/457 · set dev sets (threshold not tuned on them) (dev, development data) · judge none (deterministic) · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-15 · caveat: dev data - [F-1076] X2b on the claim kit (bounty-shaped) inert: kept/prompt identical 20/20; KIT-X2b closed (R93) · set v0.2 random20 dev (dev, development data) · judge none · n 20 · seeds UNKNOWN · CI n 20 · date 2026-10-16 · caveat: not a bounty-bar lever - [F-1078] Multiplicity check. ≈ 60 judged comparisons → ≈ 1.5 false "CI low > 0" expected; X2b z ≈ 5.5 survives Bonferroni over 100 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: main risk is selection on dev sets, not multiplicity - [F-1079] Large levers survive multiplicity correction. X2b z ≈ 5.5, LITE5-180 z ≈ 6, DEEPTHINK z ≈ 4.2 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI ≈ 60 judged comparisons · date 2026-10-16 · caveat: citation fix; ≈ 60 judged comparisons, Bonferroni over 100; the bigger risk is selection on the same dev sets - [F-1081] 1.7B + X2b vs plain 4B on pack-matched items. −0.007 (−0.031…+0.018) · set 180 self-tests dev (dev, development data) · judge Haiku · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-16 · caveat: does NOT hold on the kit: 1.7B − 4B −0.111 (−0.191…−0.047) - [F-1082] Lite gap persists with identical contexts. 1.7B X2b − 4B X2b −0.070 (−0.094…−0.044) · set 180 self-tests dev (dev, development data) · judge Haiku · n 97 · seeds UNKNOWN · CI n 97 · date 2026-10-16 · caveat: contradicts "gap is reach, not reading" | contradicts the earlier "reach, not reading" summary - [F-1083] Lite tier (Qwen3-1.7B Q4_K_M) footprint and speed. file 1,107,409,472 B; RSS ≈ 2.3 GB (peak 2.7) vs ≈ 4.0 GB 4B; desktop prefill 2.5×, decode 2.2× the 4B · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-16 · caveat: ≈ 1.2× faster with X2b at matched quality; phone unpriced - [F-1084] Lite tier quality, correctly scoped. 1.7B + X2b vs plain 4B −0.007 (−0.031…+0.018) on 180 self-tests; on the claim kit 1.7B − 4B = −0.111 and X2b inert; at identical contexts −0.070 (−0.094…−0.044) · set 180 self-tests (dev) / claim kit (dev, development data) · judge Haiku · n 180 · seeds UNKNOWN · CI n 180 / n 97 · date 2026-10-16 · caveat: "1.7B = 4B" holds only on pack-matched dev items - [F-1141] Retrieval JS on the phone: pure-JS zstd decompression ~76 % of retrieval time; native zstd ~80× faster. 499 ms of 677 ms · set UNKNOWN (status UNKNOWN) · judge n/a · n UNKNOWN · seeds UNKNOWN · CI 68 calls · date 2026-10-02 · caveat: led to native zstd arm (device-only gains) - [F-1142] Second Claude judge (Fable 5.1) vs Opus: same winner 453/456 verdicts, Pearson r 0.939, κ (correctness ≥ 4) 0.894, ratio +0.004…+0.010. 453/456 (judge.v1 ratio) · set v0.1 dev (dev, development data) · judge Opus vs Fable · n UNKNOWN · seeds UNKNOWN · CI 456 verdicts · date 2026-10-02 · caveat: both judges are Claude: tests noise, not self-preference; ≈ 23 USD list - [F-1153] Haiku compresses paired gaps: deep35 − 4B was +0.007 under Haiku vs +0.029 under Opus (×0.25). +0.007 vs +0.029 · set v0.1 dev (dev, development data) · judge both · n 57 · seeds UNKNOWN · CI n 57 · date 2026-10-07 · caveat: review later measured ×0.6-1.0 on full deep sets, ×3-4 on hard slices - [F-1166] On the 20 hardest items Haiku showed +0.029 where Opus gave +0.091; on the full set they agreed within 0.005 (+0.022 vs +0.026). ×3 compression on slice · set V-para dev (dev, development data) · judge both · n 20 · seeds UNKNOWN · CI n 20 / 51 · date 2026-10-10 - [F-1169] Criterion hashes verified: 65/65 on 2026-10-10; an outside critic found 16 of 17 matching on 2026-10-16. 65/65 (10-10); outside critic: 16/17 criterion hashes match (verdict), 19 of 20 in the C6 spot-check (1 mismatch by hashing convention, 1 without a .sha256 sidecar) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10, 2026-10-16 · caveat: some early freezes attested by mtime only - [F-1170] Thinking 4B on v0.2. Δ +0.009 · set v0.2 dev (dev, development data) · judge Haiku L1 · n 110 (inferred from set) · seeds 1 · CI −0.015…+0.035; 1 seed · date 2026-10-10 · caveat: Haiku compresses; Opus never run - [F-1184] V-para remaining loss composition (deep tier) 59 % absent knowledge + 40 % wrong facts · set V-para dev (dev, development data) · judge jev labels (LOSSMAP) · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-12 · caveat: label model unvalidated (critique C3) - [F-1187] Haiku L1 compresses paired gaps vs Opus. Haiku Δ ≈ 0.70-0.95× the Opus Δ across six pack/retrieval arms (X1 0.70, pack 0.76, ONLY 0.83, X2b 0.85, ST2 0.88, ADD 0.95); per-item Δ r 0.75-0.84 · set 180 self-tests (status UNKNOWN) · judge Haiku vs Opus · n 180 (inferred from set) · seeds UNKNOWN · CI several arms · date 2026-10-14/16 · caveat: citation fix (critique row is l.32); '0.70-0.85' are the two quoted examples; on deep/4B slices compression ranged ×0.25 to a sign flip - [F-1195] A single n=20 Haiku pass carries ±0.03–0.04 noise. per-item retest SD 0.22–0.28 rubric points · set dev (dev, development data) · judge Haiku · n 18 · seeds UNKNOWN · CI n 18–30 · date 2026-10-16 · caveat: citation fix (l.34 is the self-preference row) | at the MDE: n=20 labels are noise-level - [F-1208] Win-rate under the cross-family judge on v0.2. 4B 2.7 %, deep 6.4 % (Claude 0 %) · set v0.2 dev (dev, development data) · judge cross-family (post-hoc, different judge family) · n 110 · seeds UNKNOWN · CI n 110 · date 2026-10-17 - [F-1216] Changing the judge to another Claude model: Fable second judge. moves the ratio < 0.01 (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: citation fix (l.27 is a table header); why: does not address self-preference - [F-1231] Desktop latency per answer (p50) along the ladder on V-para. 4B 6.3 s (same backend) / 10.1 s (claim run); D 33.3/30.9 s; Q 57.5/53.2 s; H 78.9/79.4 s; G 106–109 s; T 142.9/146.0 s; Z 23.6 s · set V-para (status UNKNOWN) · judge UNKNOWN · n 51 (inferred from set) · seeds UNKNOWN · CI per seed · date 2026-10-08/10 · caveat: Mac mini M4 CPU desktop proxy, not a phone - [F-1239] Claude judge stays the headline (lower of two judges); judge family named next to every ratio; win/tie/loss leads — why: analysis-plan rule: when judges differ by > 0.05, the lower is the headline — outcome: claim v4.2.x. · date 2026-10-16/17 - [F-1245] Claim text sentence (draft v4.2.4, as of 2026-10-17) "BOAR is not shown to be half as good as frontier AI with web." · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-17 · caveat: draft, not published at time of reading - [F-1246] The desktop runner executed the real engine on 54 questions in 2 min 34 s, 0 errors. 2:34 · set 58-item bounty-style dev set (dev, development data) · judge n/a · n UNKNOWN · seeds UNKNOWN · CI 54 items · date 2026-10-01 · caveat: native parts shimmed - [F-1251] v0.3 = author writes items by hand with one non-Claude web-UI model; references from the same non-Claude model; Claude never edits item text (why: removes judge-family authorship of items and references) · date 2026-10-07 · caveat: outcome: write kit delivered 10-16; no items as of 10-17 · later: No v0.3 was written (decision D5, 2026-10-18); the v0.4 ruler set (half Claude, half OpenAI gpt-5.6-sol, frozen 2026-10-20) took its place and is not written outside the Claude family. - [F-1252] v0.3 target composition: 110 items in the same 10 strata, ≥ 40 % centred outside Europe/North America, ≤ 2 per country, each hinging on a checkable fact. 110 · set v0.3 (unwritten) (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-07 - [F-1253] v0.3 references from a non-Claude family move the denominator: v0.3 ratios are not 1:1 comparable with v0.1/v0.2. not comparable · set v0.3 (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-07 - [F-1272] PR #85 contents at opening. 13 cut 7 commits rebased + 1 test + 2 wiring commits; vitest 1,378/1,378; replay 60/60; generation 59/60 · set v0.1 public (status UNKNOWN) · judge none · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-15 10:10 · caveat: later + CodeRabbit fixes, NUMGUARD r3, HEALTHGAP r2 (tip 8ac9eb7a) - [F-1279] Retracted summary: "the small-model gap is reach, not reading" at identical contexts 1.7B − 4B = −0.070 (−0.094…−0.044) · set 180 self-tests (dev) (dev, development data) · judge Haiku · n 97, · seeds 1 · CI n 97, 1 seed · date 2026-10-16 · caveat: retracted in QUEUE l.1267 - [F-1282] v0.3 write kit handed to the author (items still unwritten) · date 2026-10-16 · later: No v0.3 was written (decision D5, 2026-10-18); the v0.4 ruler set (half Claude, half OpenAI gpt-5.6-sol, frozen 2026-10-20) took its place and is not written outside the Claude family. - [F-1289] The native app does not run the research engine's health path or deterministic answers. · date 2026-10-17 · caveat: from maintainer's own portal report + guard texts missing - [F-1297] v0.3 similarity gate: 0 pairs with word-Jaccard > 0.30 or embedding cosine ≥ 0.90 against public, safety and every held-out pool. thresholds · set v0.3 (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed - [F-1316] Public repos at release. boar-bench 139 files / 2.26 MB; boar-research 411 files / 5.53 MB · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-15 01:18 / 10:07 · caveat: one commit each, author "BOAR R&D" - [F-1353] v0.4 ruler item set FROZEN (75 items, sha f9ad2f2f...) before any system answered. · date 2026-10-20 - [F-1371] D2: submit on the 10-20 date with claim v4.2.4 (app "scored internally, final build published when it ships") post prepared ; posting not recorded in slice · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-18 · caveat: who decided: delegated coordinator; why: meets date; numbers honest; alternatives: ask extension to a later date; submit research build; reversible: partly | Verification note: The parenthetical wording ('scored internally, final build published when it ships') is the claim text's, sourced at not . - [F-1374] D5: no bench v0.3 (non-Claude held-out set) before closure. record declares no unexposed set; later partly superseded by v0.4 ruler (see §6) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-18 · caveat: who decided: delegated coordinator; why: ~14-24 h of author item writing; would land after the deadline; bar needs ~0.53 at n 110 vs best ~0.43-0.51; serves honesty, not winning; alternatives: 60-item wave 1; 6-item pilot; reversible: yes | VERIFIER NOTE: Base OF-3 says 'the author decided'; precise form: the author delegated D1-D7 and the coordinator's recommendation (skip v0.3) was adopted ( l.4). - [F-1383] Build a public ruler: true-zero utility U + task success, uncapped two-family web references, non-Claude items, submitted app on phone, gates, speed, install. led to v0.4 items + spec v1 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-19 · caveat: who decided: coordinator from 8-agent workflow (both critics converged); why: five misreadings of the bounty in the old bench; alternatives: keep old bench as headline; reversible: yes - [F-1389] Ruler headline weights the 11 health_safety items at 0.5, adds a safety gate, publishes weight-1 U beside it. in spec v1 §2; reviewer F3 flags timing (decided after items existed) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: who decided: author; why: bounty asks for casual lookup/research, never names health; alternatives: weight 1; reversible: yes AI-reviewed, not clinically reviewed. - [F-1394] Honest Detailed prompt (D1) cuts wrong-claim rate on the app's chat-template path. 0.231 -> 0.062 (paired -0.169) · set set A (agent-written chat turns), dev (dev, development data) · judge Claude Opus sentence audit; wrong claims / claims · n 15 · seeds UNKNOWN · CI CI -0.281..-0.073; n 15 answers/arm, 407 claims · date 10-18 · caveat: answers 42 % shorter; vaguer on 2/15 - [F-1397] D4 costs no judged ratio on v0.1. D4 0.427 vs D1 0.425 (+0.003); D0 0.452 (judge.v1 ratio) · set v0.1 public dev (dev, development data) · judge Claude Haiku 5.5-low, raw ratio · n 57 · seeds 1 · CI CI -0.024..+0.028; 57 ids, 1 seed · date 10-18 · caveat: win 0/57 for both. Proxy contexts (not the app retrieval), one seed, Claude Haiku judge; raw ratio only (the source has no floor-adjusted value). - [F-1401] PACKREAD: reading with packs, D4 prompt. 4B 0.650, LFM2.5 0.554, 1.7B 0.529; engine Succinct anchor 0.610 · set pack self-tests, dev (dev, development data) · judge Claude Haiku, raw ratio · n UNKNOWN · seeds UNKNOWN · CI 54 pack self-test items; LF-D4 vs M4-D4 -0.096 (-0.145..-0.052) · date 10-19 · caveat: identical contexts: 1.7B gap is reading - [F-1406] V-para 4B gap decomposition. gap 0.586 ratio points: wrong 0.309 (23 items), absent 0.239 (24), thin 0.038 (4), refusals 0 (judge.v1 ratio) · set V-para (dev-style since 10-09) (dev-style since 2026-10-09, development data) · judge Claude Opus judge.v1; LOSSMAP labels · n 51 · seeds UNKNOWN · CI n 51, s42 · date 10-19 · caveat: single-seed labels, shares ±0.05 - [F-1407] Facts for V-para are mostly nowhere. in installed kit 6/51; all packs on disk 10/51; full English Wikipedia 9/34 absent items · set V-para (dev-style since 2026-10-09, development data) · judge coverage counts · n 51 · seeds UNKNOWN · CI n 51 / 34 · date 10-19 · caveat: The 34 are the deep tier's absent items; the 4B's absent set is 24, of which 17 are shared. - [F-1411] Estimated ceiling of all 8 GB levers on V-para. about +0.01-0.02 raw (judge.v1 ratio) · set V-para (dev-style since 2026-10-09, development data) · judge estimate · n UNKNOWN · seeds UNKNOWN · CI estimate, untested · date 10-19 · caveat: estimate only - [F-1416] v0.4 candidate writers. Claude 36, OpenAI gpt-5.6-sol 34; 302/307 points supported, 5 disputed · set v0.4 drafts (frozen ruler set (held-out for the ruler; counts only)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 10-20 - [F-1418] Pixel 8a speed, 1.7B vs 4B. 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s; 4B LMK kill at 4.4 GB RSS · set public dev questions (dev, development data) · judge timing · n UNKNOWN · seeds UNKNOWN · CI 9-question runs · date 10-20 · caveat: 4B 73/73 only after a reboot | VERIFIER NOTE: Pre-BATCHRET, 9 public dev questions; not comparable with the 73-question p50 of 10-21. - [F-1422] BATCHRET. SQL calls per retrieve 937 -> 48; retrieval byte-identical 390/390 · set public dev (dev, development data) · judge parity · n 390 · seeds UNKNOWN · CI 390 items · date 10-20 - [F-1423] BATCHRET on the Pixel 8a, 1.7B. TTFT 9.12 -> 5.55 s (-39 %); retrieval 5.04 -> 1.45 s; total 12.39 -> 9.40 s; safety card 8.27 -> 3.10 s · set litelat-9 (unspecified) · judge timing · n 9 · seeds UNKNOWN · CI 9 items, A B A B, thermal 0 · date 10-20 - [F-1430] REFUSE-UNSUPPORTED. visible confident errors 7 -> 3 of 177 under Claude Opus (re-judge of the 38 deciding answers; Claude Haiku 10 -> 4); false refusals 0/15; U on answered +0.030 (+0.005..+0.057, Haiku); coverage 0.435 -> 0.328; +9.3 s p50 projected on the Pixel 8a · set MAIN 97 + 80 self-tests, dev (dev, development data) · judge Claude Haiku 5.5-low; Claude Opus re-judge for the confident-error count · n 177 · seeds UNKNOWN · CI n 177 · date 10-20 · caveat: GATELAT: no cheaper variant keeps AUC | Verification note: Judge-dependence: Haiku 10 -> 4 of 177; Opus re-judge of the 38 deciding answers 7 -> 3 of 177 ; the claim text should quote 7 -> 3 with Opus named. +9.3 s is projected. - [F-1431] Judge v2 (binary checklist) test-retest. CE test-retest 0.966 vs v1 0.924 (needed 0.974); SD 0.80x (needed 0.60x) -> NO-GO · set dev (dev, development data) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI frozen rule · date 10-20 - [F-1432] Haiku vs Opus on confident-error labels. agree 39/47 (kappa 0.63); Haiku over-calls; Opus +0.106 more lenient on correctness · set dev (dev, development data) · judge UNKNOWN · n 47 · seeds UNKNOWN · CI n 47 · date 10-20 - [F-1529] RULING R1: judge = judge.v1 unchanged + DECLINE2STAGE zero per detected non-answer; D 0.058 kept as sensitivity row. adopted into ruler r3/r4 (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: delegated coordinator (override of a frozen FAIL, accepted with conditions by reviewer); why: judge v1.1/v1.1b changed CE verdicts (5-7/40 flips); DECLINE2STAGE's 2 bar-(b) hits were engine refusal strings; alternatives: judge v1.1; decline subtraction only; reversible: yes (pre-exposure) - [F-1536] Final score = submitted native APK on Pixel 8a, v0.4 frozen (75 items), Claude + OpenAI gpt-5.6-sol judges, frontier + web references. ruler r3/r4 built for it; run pending a new RC · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: author (GRILL); why: the bounty judges the app; alternatives: engine-only score; reversible: partly (depends on RC) - [F-1537] Submission = the maintainer's native Kotlin app, not the RN bench engine. re-based speed lines (bench levers page-only) and claim section 0 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: author; why: that is what will be submitted; alternatives: engine-only claim; reversible: no (fact of the submission) - [F-1543] Freeze r4: add information-available split (FACT-AVAIL) + ORACLE/ORACLE-0 rows; U-total stays verdict. frozen sha 82772e33; labels quarantined (ruler U-total) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: author asked; coordinator + reviewer (H1-H8); why: separate "packs lack it" from "app fails to deliver it"; alternatives: no split; reversible: no (frozen) Metric: ruler v0.4 U-total (ratio of sums against the reference mass, 71 headline items, health weight 0.5); never compare with the judge.v1 ratio. - [F-1565] Phone, Pixel 8a, 73 questions, bench app. 1.7B p50 4.2 s, p90 12.2, 12.4 tok/s, PSS 3.39 GB; Gemma E2B p50 11.8 s, 7.0 tok/s, PSS 4.52 GB · set v0.1 (dev) (dev, development data) · judge speed · n 73 · seeds 1 · CI n 73, 1 run each · date 10-21 03:31 · caveat: p50 is over all 73 items including deterministic-layer and declined items: the 1.7B made 38 LLM calls vs Gemma 55 and declines 35 % of v0.1; model-path answer p50 is about 12 s. Not like-for-like with the 9-question 13-16 s runs of 2026-10-20. - [F-1593] Where the fact arrives. U 0.157 → 0.330, dU +0.173 (+0.082..+0.266); correct 13.6 → 50.0 per 100; decline 22.7 → 0 · set cooking/money self-tests (dev) (dev, development data) · judge Claude Opus U · n 22 · seeds 1 · CI n 22 units, seed 42 · date 10-22 22:32 · caveat: selected slice (newly delivered) - [F-1601] Public pack uploads in slice. simplewiki r2 (convert fix) 10-21; kb-index-research snapshot 10-22 · set UNKNOWN (n/a) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 10-21/22 · caveat: standing pack-workflow OK - [F-1628] Earlier: "the ruler was frozen on 2026-10-20". Now: on 2026-10-20 the v0.4 item set was frozen; spec r2 was a draft (its binary-checklist judge v2 was NO-GO); RULER-SPEC v1 r3 (2026-10-21) is the first frozen spec, r4 adds the information-available split. Changed to: Ruler = RULER-SPEC v1 r3/r4: judge.v1 + DECLINE2STAGE, two families, A-union verdict arm, FACT-AVAIL split (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: Verification note: Earlier standing is from slice u-1820, not the base: on 10-20 the v0.4 ITEM SET was frozen (not 'the ruler'); spec v1 r2 (binary checklist judge) was a draft, judge v2 already NO-GO . r3 is the first frozen spec. - [F-1633] Earlier: Product behaviour "refuse + show anyway" (GRILL D4). Changed to: Show-anyway only behind second confirmation, never on health; RC1 has no warning line at all. - [F-1695] Shaped factbook v2: card drafted as user-selected only; publish waits for author OK. Not published as of 10-24; in ENGINE-RC kit; BoarLite D3 puts it in the APK · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-23 · caveat: who: delegated coordinator; publish = author; why: Opus +0.121 but one new warned CE; correct-any 0.50 < 0.60 eligibility; reversible: yes · later: 2026-10-24 18:03: BoarLite keeps factbook v1; the shaped factbook v2 is not in the app. - [F-1701] Closed simple chat app for one way of use: Succinct fixed; Deep Research, Detailed and length setting removed; free follow-ups; warning line kept; sources shown. Applied on a local branch · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 · caveat: who: author; why: small models gave the best results this way; reversible: yes (product) - [F-1703] BoarLite = a NEW bounty submission by the author (not the app maintainer's app); closed app, everything bundled, full K13 kit in one APK, shaped factbook v2, port the warning line, Qwen3-1.7B fixed. Implementation next; APK ≈ 3.0 GB, hosting open · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 · caveat: who: author; why: (decision page); reversible: partly (submission identity) · later: 2026-10-24 18:03: factbook v1 stays (D3 reverted) and a fixed line on every answer replaces the ported warning line (D4); final candidate b4ef94f3 at 19:40. - [F-1705] Correct-any when own article is kept vs not. 0.65 (n 98) vs 0.28 (outcome-label share) · set 180 pack self-tests (written from the packs), dev (development data) · judge Claude Haiku labels · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-23 · caveat: "about 2 in 3 when delivered" | 'kept' = own article kept (article level), between-item split, n 98 kept. - [F-1706] Own article delivered (kept) 54 % ; cited 28 % · set 180 pack self-tests (written from the packs), dev (development data) · judge 0-LLM funnel · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-23 · caveat: delivery = bottleneck | 'kept' here is article-level (own article in the kept list), not the chunk-level delivery used for MAIN facts since 10-22 ; do not mix with the 42/92 chunk-level count. - [F-1708] Without the pack: wrong-confident / wrong-warned / honest-unknown. 0.17 / 0.45 / 0.12; refused-wrongly 0 · set 180 pack self-tests (written from the packs), dev (development data) · judge Claude Haiku · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-23 · caveat: honesty comes from the warning line, not from abstention - [F-1709] UXMATRIX Succinct: correct-any ON / OFF. 0.35 / 0.28 (CWS 0.28 / 0.03; WC 0.22 / 0.07) (outcome-label share) · set 60 dev items (40 cooking/money, 15 delivered general, 5 no-pack) (dev, development data) · judge Claude Haiku; outcome shares · n 60 · seeds 42 · CI n 60; seed 42 · date 2026-10-23 · caveat: Succinct and Detailed on different hosts Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-1710] UXMATRIX Detailed: correct-any ON / OFF. about 0.27 / 0.17 (headline rounding; the CWS + CFM cells sum to 0.26 / 0.16); WC OFF 0.15 (outcome-label share) · set 60 dev items (40 cooking/money, 15 delivered general, 5 no-pack) (development data) · judge Claude Haiku · n 60 · seeds 42 · CI n 60; seed 42 · date 2026-10-23 · caveat: Deep arms not run (scope cut) Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-1711] With pack off the model never says "I don't know" itself; all HU are engine declines; warning line shown OFF. 75 % (Succinct), 63 % (Detailed) · set 60 dev items (40 cooking/money, 15 delivered general, 5 no-pack) (development data) · judge Claude Haiku + 0-LLM · n 60 · seeds UNKNOWN · CI n 60 · date 2026-10-23 · caveat: 6 of the declines are health false alarms on cooking/money AI-reviewed, not clinically reviewed. - [F-1712] WARN-KEPT: confident errors with pack on all keep a citation; lexical rule warns 9/26 at 5/37 correct cost. NO · set UXMATRIX traces + MAIN 97 (dev, development data) · judge 0-LLM · n 26 · seeds UNKNOWN · CI n 26 / 37 · date 2026-10-23 · caveat: errors are misreadings of a relevant chunk - [F-1714] NOWIKI: kit without Wikipedia packs. 42 -> 33 MAIN facts; 10 public v0.1 items lost; topics anchor 25 -> 11; health: 1 self-test + 1 topics item lost (MAIN health losses 0) · set dev (dev, development data) · judge 0-LLM · n 97 · seeds UNKNOWN · CI n 97 / 180 / 27 · date 2026-10-23 · caveat: verdict KEEP AI-reviewed, not clinically reviewed. - [F-1715] QREWRITE oracle ceiling (answer-form query) MAIN 42→51 (+9 < +10); self-tests kept 100→118 (+18 < +20) · set dev (dev, development data) · judge 0-LLM · n 97 · seeds UNKNOWN · CI n 97 / 180 · date 2026-10-23 · caveat: oracle uses answer terms the user lacks - [F-1718] FACTBOOK-V2 Opus confirm. U 0.316→0.436 pooled, +0.121 · set factbook self-tests, dev (dev (pack-written self-tests), development data) · judge Claude Opus judge.v1 (ratio U, old family) · n 20 · seeds UNKNOWN · CI CI +0.030..+0.221; n 20; 2 seeds · date 2026-10-23 · caveat: NOT-CONFIRMED: 1 new warned confident error - [F-1720] RETBENCH best (k4 + compress 0.4): MAIN 42→45. closed by probes (2 dose/number changed) · set dev (dev, development data) · judge 0-LLM · n 97 · seeds UNKNOWN · CI n 97 / 65 · date 2026-10-23 · caveat: RETBENCH v1 = reusable 0-LLM bench AI-reviewed, not clinically reviewed. - [F-1722] ENGINE-RC reading cost (S8) 21.26 MB/question vs 125.09; ctx p50 0.978x; disk 1.06x · set UNKNOWN (status UNKNOWN) · judge 0-LLM · n UNKNOWN · seeds UNKNOWN · CI quiet desktop runs · date 2026-10-23 · caveat: desktop only, phone unmeasured - [F-1725] False "Not found in this phone's offline library" line with sources kept. 37 of 97 answers showed "Not found" although passages were kept; after the fix 0 do (1 correct "Not found" remains, on an item that keeps no source) · set MAIN dev (dev, development data) · judge 0-LLM · n 97 · seeds UNKNOWN · CI n 97 · date 2026-10-23 - [F-1726] Night generation. 25 jobs, 2,855 answers, 0 failures, 0 judge calls · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-23 · caveat: judge picked by author next day - [F-1728] S+ / two-paragraph personas. correct-any ON 0.42 / 0.42; WU ON 0.23 / 0.25 (vs 0.12) (outcome-label share) · set UXMATRIX 60, dev (development data) · judge Claude Haiku · n 60 · seeds 42 · CI n 60; seed 42 · date 2026-10-24 · caveat: warning line 0.60→0.48 with longer answers Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-1729] Follow-ups on ENGINE-RC, with and without subject carry. correct-any 0.458 both; on-referent 1.00; turn-1 source 0.083→0.708; WU 2→4 of 48 (outcome-label share) · set F8 dev (dev, development data) · judge Claude Haiku · n 48 · seeds 42 · CI n 48; seed 42 · date 2026-10-24 · caveat: health follow-ups 0.19 both Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. AI-reviewed, not clinically reviewed. - [F-1836] Adopt the robust ruler design (true-zero U, uncapped two-family refs, items outside the judge family, app on phone) became RULER-SPEC v1 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-19 · caveat: who: delegated coordinator (8-agent workflow); why: five misreadings of the bounty in the old bench; reversible: yes - [F-1864] BoarLite = a new bounty submission by the author (not the app maintainer's app): no INTERNET permission, full K13 kit in one APK (~3.0 GB), shaped factbook v2, warning line ported, Qwen3-1.7B fixed; English-only UI (from the clean build decisions) implementation pending; hosting of a >2 GB APK open · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 · caveat: who: author; why: closed app matches what was measured; reversible: partly (new claim) · later: 2026-10-24 18:03: factbook v1 stays and a fixed line replaces the ported warning line; final candidate b4ef94f3 at 19:40. - [F-1908] MODEL-USAGE audit: Is the measurement pipeline sound? 33 findings, 6 P0 (stale CE metric; refusals labelled CE; unchecked X2b padding; unreachable screen bar; Qwen3.5-2B unscreened) · set UNKNOWN (status UNKNOWN) · judge review · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: criterion frozen: n/a (7 finders + 2 refuters); arm verdict: follow-ups launched - [F-1931] CITEFIX: Is the citation-drop failure a format issue? engine's 50 % key-word support check strips 161 paraphrased citations; format fix flips no gate · set 910 candidate answers (status UNKNOWN) · judge 0 Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: yes (93c85722); arm verdict: open lever: paraphrase-tolerant check - [F-1933] MIN3CITE: Ministral-3-3B + citation prompt v2. stage 2 Haiku +0.062 (+0.037..+0.089); Opus +0.047 (+0.021..+0.075); ~3x phone time; −0.037 vs 4B (Opus, 61) · set screens; MAIN 97 K0 (dev, development data) · judge Claude Haiku; Claude Opus revisit · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: yes (a4d2d9a3; revisit e9813e2c); arm verdict: Haiku PASS; closed as dominated - [F-1955] JUDGE-V1_1 / V1_1B: Can one rule line make judge.v1 give declines a true zero? declines U 0.000, but CE flips 7/40 and 5/40; retest SD 0.0265/0.0295 vs 0.0206 (judge.v1 ratio) · set decline variants; 40 controls (status UNKNOWN) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: yes (bf560ea9; 6edbacf6); arm verdict: NO-ADOPT both - [F-1957] DRIFT17: Does the 1.7B degrade over a session? CE-unwarned t1 0/10 vs t4-5 3/20, CI covers 0; one unsafe health error in both arms · set longsession-v1 80 (dev) (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI rise CI 0.00..0.30 · date 2026-10-21 · caveat: criterion frozen: yes (5ea4b345); arm verdict: KILLED (no drift detected) AI-reviewed, not clinically reviewed. - [F-1960] RV4-CITY-BOUND: How much can the 7 current-city items swing v0.4? 7 items = 10.7 % of headline weight; swing ≥ 0.107 U · set v0.4 tags (held-out, script) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge arithmetic · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: yes (53f75edc); arm verdict: KILL branch: separate stratum scored as given - [F-1965] PK-3 OVERLAP: Is the self-test pack gain a lexical-overlap artefact? reweighted to real overlap +0.064 (+0.035..+0.095) · set cand80 vs 37 real golds (status UNKNOWN) · judge reweighting · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: yes (fe44d05e); arm verdict: KILL (not an artefact; question type/reading) - [F-1989] CONVERT-FIX: Converter drops unit-conversion values. artefact chunks 3,176→524; simplewiki r2 published 15:01 · set pack rebuild (status UNKNOWN) · judge counts · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: n/a (bug + 11 tests); arm verdict: shipped (pack r2) - [F-1998] STYLE15B: Succinct vs Detailed on 1.5B and 1.7B. 1.5B 0.220 vs 0.133; 1.7B 0.263 vs 0.224; CE shown halves · set v0.1 51 (dev, development data) · judge Claude Haiku; Claude Opus diagnostic 11 · n UNKNOWN · seeds UNKNOWN · CI 1.5B +0.087 (+0.038..+0.143); 1.7B +0.038 (−0.005..+0.084) · date 2026-10-21 · caveat: criterion frozen: yes (474ac0a0); arm verdict: Succinct NONINFERIOR - [F-2025] SEFIX-124: F1/F2/F4 without F3. chunk-level 42/92, 0 lost; pooled +0.013 · set MAIN 97, 2 seeds (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI +0.013 (−0.008..+0.035) · date 2026-10-22 · caveat: criterion frozen: yes (07ea2b5e); arm verdict: not a lever-candidate; adopted for delivery - [F-2042] DEEPMODE: Deep Research multipass on the 1.7B. U 0.174 vs Detailed 0.224 vs Succinct 0.263; 2.6-5.6x slower · set 27 topics; v0.1 51 (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI vs Detailed −0.050 (−0.087..−0.015) · date 2026-10-23 · caveat: criterion frozen: yes (8753e6b7); arm verdict: worse; hide - [F-2072] Six prompt/decoding/confidence levers for the 1.7B were null. fixes 1-4 of 21-23 CEs (need 6); logprob AUC 0.530; first-token AUC 0.488 · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI screens of 37 · date 2026-10-20 · caveat: old CE metric; REREAD re-counted - [F-2073] Null arms already "fix" a quarter of screen problems. 23-31 %; bar = max null net + 2 · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 2 nulls per set · date 2026-10-20 · caveat: calibration rests on 2 nulls - [F-2074] Groundedness gate refusing unsupported 1.7B answers. visible CEs 10→4 of 177 (Opus 7→3); false refusals 0/15; +9.3 s projected per answer · set MAIN 97 + 80 self-tests (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku; Claude Opus · n 177 · seeds UNKNOWN · CI n 177, 1 seed · date 2026-10-20/21 · caveat: latency blocks shipping; aLoRA unverified - [F-2076] Gemma 4 E2B vs Qwen3-1.7B on MAIN 97. Haiku +0.028 (+0.006..+0.052) FAIL by 0.002; Opus +0.031 (+0.005..+0.058) GO by 0.001; pooled Opus +0.036 (+0.011..+0.062) · set MAIN 97 (dev) (dev, development data) · judge Claude Haiku; Claude Opus · n UNKNOWN · seeds UNKNOWN · CI 2 seeds · date 2026-10-21 · caveat: gain mostly fewer refusals; 2.8x phone time - [F-2078] Pixel 8a 73-question runs. 1.7B p50 4.2 s, PSS 3.39 GB; Gemma 11.8 s, 4.52 GB; 4B 26.0 s complete, PSS peak 4.76-4.92 GB · set public dev (dev, development data) · judge timing · n UNKNOWN · seeds UNKNOWN · CI 73 items · date 2026-10-21 · caveat: 1.7B declines 35 % of v0.1 (not like-for-like) | Runs differ: 4B 26.0 s complete is final-cut7-v01 (pre-BATCHRET engine); the 4B LMK kill is the 10-20 first attempt . 1.7B declines 35 % of v0.1, so speeds are not like-for-like . - [F-2080] Pack self-test gain does not reproduce on real items. self-test fact-added cell +0.139 (+0.075..+0.201) n 27 vs real −0.010 (−0.077..+0.050) n 10 · set self-tests (pack-written) vs MAIN keyed (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 1 seed · date 2026-10-21 · caveat: post hoc; small n on real - [F-2081] judge.v1 does not give declines a true zero. bare refusal U 0.058 (0.037-0.081); hedge 0.053; fluent no-content 0.016 (judge.v1 ratio) · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 30 v0.1 ids · date 2026-10-21 · caveat: leak in correctness dimension - [F-2083] Haiku vs Opus calibration. κ CE 0.75, κ correct 0.49 (warned 0.22); Opus finds 98 correct Haiku misses vs 10 the other way; U shift +0.022 · set dev (dev, development data) · judge both · n UNKNOWN · seeds UNKNOWN · CI 1,911 units · date 2026-10-21 - [F-2084] Show-anyway reveals under Opus. MAIN census 11 correct : 16 CE (0.69; non-health 0.56, CI 0.22-1.35) · set MAIN 97 (dev) (dev, development data) · judge Claude Opus · n UNKNOWN · seeds UNKNOWN · CI 58 hidden answers · date 2026-10-21 · caveat: Haiku 7:19 on same rows AI-reviewed, not clinically reviewed. - [F-2085] Warning line precision/recall (1.7B) precision 0.39 (0.32-0.46), recall 0.67 (0.59-0.75); Opus 0.40 / 0.65 · set dev (dev, development data) · judge Claude Haiku; Claude Opus · n UNKNOWN · seeds UNKNOWN · CI 746 answers · date 2026-10-21 · caveat: mostly a support signal (77 % of warned rows had a source) - [F-2092] Question form decides retrieval. hit@kept base 50 %, keywords 62 %, yes/no 83 %, entity-free 10 %, situational 25 %, follow-up 8 % · set dev, agent-written (dev, development data) · judge replay · n UNKNOWN · seeds UNKNOWN · CI 48 facts × 9 forms · date 2026-10-21 · caveat: follow-up hole is a bench-engine artefact - [F-2093] When the reference chunk is kept, the 1.7B answers near reference level. U 0.876 (n 43, CE 2) vs 0.185 (n 147, CE 75) · set pack facts (dev, pack-written) (dev (pack-written self-tests), development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 192 answers, 1 seed · date 2026-10-21 · caveat: between-item comparison; real items flat (F-u-results-34) | QFORM items: 48 facts from public self-tests (factbook, firstaid, medlineplus) plus facts authored from pack text (crypto, wikivoyage, visa) x forms; Haiku judge.v1 ratio U, not an outcome label . Never print next to PACKPROOF correct-any as the same metric. AI-reviewed, not clinically reviewed. - [F-2095] Chunk-level delivered facts on MAIN 92. N0 21; gidf 28; SEFIX-124 42; SEFIX-ALL 47 · set dev (dev, development data) · judge counts · n UNKNOWN · seeds UNKNOWN · CI replays · date 2026-10-22 · caveat: article-level counts overstated by 8-13 - [F-2096] SEFIX-ALL end-to-end. Haiku pooled +0.023 (+0.001..+0.048) over 97; Opus CV5 −0.002 (−0.062..+0.061); Opus where fact delivered (cm) +0.173 (+0.082..+0.266) · set MAIN 97 (dev) (dev, development data) · judge Claude Haiku; Claude Opus · n 42 · seeds UNKNOWN · CI 2 seeds / n 42 / n 22 · date 2026-10-22 · caveat: withdrawn: includes unsafe F3; SEFIX-124 alone +0.013 (−0.008..+0.035) AI-reviewed, not clinically reviewed. - [F-2097] F3 health fallback is unsafe. fires 21/65 probes; unsafe 9 → 13 · set dev (dev, development data) · judge blinded agent reading · n UNKNOWN · seeds UNKNOWN · CI 65 probes · date 2026-10-22 · caveat: AI-reviewed AI-reviewed, not clinically reviewed. - [F-2098] On real items, delivering the fact does not separate answers. delivered 0.420 vs not 0.419 (+0.001, −0.071..+0.069); within-item +0.044 (+0.009..+0.082) · set dev (dev, development data) · judge Claude Haiku; Claude Opus slice +0.013 · n UNKNOWN · seeds UNKNOWN · CI MAIN 92, 2 seeds · date 2026-10-23 · caveat: page must state plainly - [F-2102] Sentence support on the app's 27 topics. unsupported Detailed 0.468 vs Succinct 0.283; invented specifics 98 vs 7 · set dev (dev, development data) · judge Claude Haiku sentence audit · n UNKNOWN · seeds UNKNOWN · CI 27 topics · date 2026-10-23 - [F-2103] W78 stack reading cost. 125.09 → 21.26 MB per question; cold 620 → 92 ms; kept 96/97; facts 42/42 · set dev (dev, development data) · judge replay · n UNKNOWN · seeds UNKNOWN · CI MAIN 97 · date 2026-10-23 · caveat: phone unmeasured - [F-2104] Shaped factbook. fact chunk kept 7→15/20; correct 5→10/20; Opus U 0.316→0.436 (+0.121, +0.030..+0.221) · set factbook self-tests (pack-written) (dev (pack-written self-tests), development data) · judge Claude Haiku; Claude Opus · n UNKNOWN · seeds UNKNOWN · CI 20 items, 2 seeds · date 2026-10-23 · caveat: NOT-CONFIRMED (1 new warned CE); does not transfer to the native engine - [F-2105] ENGINE-RC passes the 0-LLM validation suites. identity 97/97 & 180/180; MAIN delivered 42; ST fact 84 vs 77; routes 103/110; 21.26 MB/q · set dev (dev, development data) · judge replay · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-23 · caveat: V1 letter-FAIL pre-existing - [F-2106] Longer styles do not add correctness. Succinct WU ON 0.12 vs S+ 0.23, P2 0.25; correct-any 0.43 / 0.42 / 0.42 (outcome-label share) · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 60 items · date 2026-10-24 · caveat: Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio. - [F-2107] Follow-up accuracy is reading-limited. correct 0.458 with or without subject carry; source kept 0.08 → 0.71 with carry · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 48 F8 · date 2026-10-24 - [F-2108] Bench v0.4 exposure. first answers by two engines 10-24; any later engine change = developed after exposure (ruler U-total) · set v0.4 (held-out ruler, now exposed) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Opus · n UNKNOWN · seeds UNKNOWN · CI 75 items × 2 seeds × 2 engines · date 2026-10-24 · later: 2026-10-24 20:34: a third engine (BoarLite b4ef94f3) answered v0.4 in the final ruler run; its numbers are internal pending the author. - [F-2109] Ruler spec frozen. r3 sha 0fac2286 (10-21 20:48); r4 sha 82772e33 (21:18) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: private pre-registration - [F-2110] Claude references for v0.4 frozen before any app answer. 75/75 items, 88 turns, uncapped, median 703 words, median 6 sources · set v0.4 (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Opus + web · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: OpenAI gpt-5.6-sol half 15/75 by 10-24 - [F-2111] Information-available labels on v0.4. AVAILABLE 19 / PARTIAL 25 / NONE 27 (strict 15/23/33) of 71 · set v0.4 (quarantined labels) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Haiku + Claude Opus · n UNKNOWN · seeds UNKNOWN · CI validation 34/40 · date 2026-10-21 · caveat: counts only; lists stay quarantined - [F-2113] Current-city stratum weight. 7 items = 10.7 % of headline weight (7/65.5) · set v0.4 tags (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge arithmetic · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 - [F-2114] Reference draw noise. two uncapped Opus draws move U ≤ 0.015 · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 20 v0.1 · date 2026-10-21 · caveat: between-family variance unmeasured - [F-2115] Same-family judge lift in the literature. 3.4-8.4 pp (open-model families; Claude/GPT not tested) · set external (external) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: external paper, abstract only - [F-2117] Model-onboarding pattern: citation-dropping models get penalised by the engine. Ministral, Qwen3.5-2B, SmolLM3, granite, FOTAPOL all drop [n]; false refusals up to 9/20 · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI screens · date 2026-10-20/21 · caveat: engine-specific confound (lexical support check) - [F-2118] Expert-streaming engine cross-built for Android. builds; phone numbers unmeasured · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-19 · caveat: no phone run in this slice - [F-2173] Earlier: Submission = the maintainer's native Kotlin app (10-15 decision). Now: 10-24: BoarLite, a new closed offline app by the author, is the bounty submission. - [F-2189] Author decisions for BoarLite (18:03): port a minimal health path (the fixed safe text plus the emergency number when no health-family source is kept); replace the ported warning line (D4) with one fixed, discreet line on every answer; keep factbook v1 in the app (D3 reverted); port the bench search fixes (SEFIX-124 + HEADPASS-X2b) before the submission under a frozen criterion. D3 reverted (factbook v1 stays); D4 replaced by a fixed line; health path + search port to build · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 18:03 · caveat: who: author (decision page built on the native-engine comparison, whose numbers are internal-pending-author, PD-09); reversible: yes - [F-2191] BoarLite health gate, first version (fb6ffbf6): the fixed safe text shows on 8 of 65 safety probes vs 41 on ENGINE-RC; the app's lexical health test missed 29 of ENGINE-RC's 41 because it does not fire on drug, dose or vital-sign wording, and none of the 4 answers that invented a dose figure was caught. 8 / 65 fixed text (ENGINE-RC 41 / 65); 29 missed · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 18:12 · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2192] Health detector B+FU (prototype): the app's lexical health test OR 11 lexical rules (drug names, dose units and events, vitals, child fever temperature, symptoms, poisoning, alcohol, pregnancy, child dosing, trauma, altitude, diving) OR a short follow-up of a health turn; it gates 37 of 65 probes, including all 4 answers that had invented a dose figure, with 0 new fires on 251 non-health dev items and 1 of 40 adversarial items. 37 / 65 gated; 4 / 4 invented-dose answers; new false fires 0 / 251 and 1 / 40 · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes; 251 + 40 non-health items · seeds UNKNOWN · CI not computed · date 2026-10-24 18:23 · caveat: Rules fitted on the same 65 probes (in-sample). Rejected: title overlap with health articles (gated 5 of 19 health self-tests) and a port of the mmBERT health detector (106 MB, 2-3 days). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2193] B+FU landed in BoarLite (a9ca9dd1): with retrieval, 41 of 65 probes get the fixed text, including the 4 invented-dose answers and 12 probes beyond ENGINE-RC's 41; on 60 health self-tests the predicate fires on 19 and the fixed text shows on 12, all 12 with only a Simple English Wikipedia or Vital-articles article kept, which the gate does not count as a health source; 2 of 60 UXMATRIX cooking items (food poisoning, an allergy) also get the fixed text. 41 / 65 probes; 12 / 60 health self-tests; 2 / 60 UXMATRIX items · set 65 F3-SAFETY probes (dev); 60 health self-tests (pack-written); UXMATRIX 60 (dev, development data) · judge none (0 LLM) · n 65 probes; 60 health self-tests; 60 UXMATRIX items · seeds UNKNOWN · CI not computed · date 2026-10-24 18:29 · caveat: Follow-up check: 12 fires on 96 follow-up turns (6 by the follow-up rule, all health follow-ups); 2 follow-up-only fires on 50 long-session turns (one debatable). The 12 Wikipedia-only self-tests are the Wikipedia-health limitation (see the 19:32 decision). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2194] The health gate moved after retrieval (9d7fdc2e), so no-search tasks are gated too (a blood-pressure reading routed as a calculation now gets the fixed text): 42 of 65 probes get the fixed text; the missed ones include 7 probes with a health passage kept and 3 plain unit conversions. 42 / 65 fixed text · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 18:31 · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2195] SEARCHPORT baseline B0 (native engine 078a2627 with the APK's pack kit and factbook v1, 0 LLM): 26 of 92 keyed MAIN facts delivered at chunk level; own fact chunk kept on 38 of 180 pack self-tests; kept-context p50 347 tokens; retrieval p50 211 ms on a laptop; recorded before any port code was evaluated. MAIN 26 / 92; ST 38 / 180; context p50 347 tokens · set MAIN 92 keyed (dev); 180 pack self-tests (pack-written) (dev, development data) · judge replay (0 LLM) · n 92 keyed MAIN items; 180 self-tests · seeds UNKNOWN · CI not computed · date 2026-10-24 18:34 - [F-2196] SEARCHPORT P1 (the bench search fixes SEFIX-124 + head pass + X2b appends, ported to the native engine) failed its frozen safety gate (d) after the one allowed fix: delivery MAIN 26 → 40 (+14, 0 lost), own fact chunk 38 → 64, 0 health-family losses, context p50 1.07×, but 3 probe answers gained a dose figure copied from an appended first-aid or health Q&A passage; the port stays in the code with its switch off. MAIN 26 → 40; ST 38 → 64; context 1.07×; 3 new dose-figure probes · set MAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev) (dev, development data) · judge replay (0 LLM); regex count on answers · n 92 keyed MAIN; 180 self-tests; 65 probes · seeds 1 (42) for the probe answers · CI not computed · date 2026-10-24 18:34 · caveat: Ported unchanged, the appends doubled the native prompt (context 2.33×, gate c); the one fix compressed appends with the retrieved passages within 1.2× (MAIN 47 → 40). Criterion frozen before any port code (sha fa3b1144). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2198] Device smoke of BoarLite (APK 9d7fdc2e) on a Pixel 8a in airplane mode: first launch unpacks 18 files (3 GB) to the chat in 14 s; 0 bytes received or sent over the network; 0 crashes; PSS 3.09 GB after 5 answers; a child ibuprofen-dose question gets the fixed health text; the first answer took about 2 min (model load), later answers 5.9-7.4 s at 15.9-16.0 tok/s. unpack 14 s; network rx 0 B / tx 0 B; PSS 3.09 GB; 0 crashes · set device smoke (hand questions, Pixel 8a, airplane mode) (dev, development data) · judge none (hand check) · n 5 questions · seeds UNKNOWN · CI not computed · date 2026-10-24 18:45 · caveat: One device, 5 hand questions. Findings: a unit conversion listed unrelated sources (fixed later by QuickMath); no loading indicator for the first answer (fixed later by the model preload); retrieval noise on one answer. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2199] Author approves the BoarLite onboarding: an unpack screen ("Everything BoarLite knows is inside this app. Nothing is downloaded.") followed by a 5-card tour with a live status strip (unpacking, loading the model, ready); Open is enabled only when the model is loaded; downloads, model choice, answer style, Deep Research and language are removed from setup. approved · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 18:51 · caveat: who: author; reversible: yes - [F-2200] QuickMath (43b50410): pure unit conversions and arithmetic are computed in code with no model, search, sources or line; health-gated turns and turns after a health turn are never computed. Model preload starts right after unpack (b8df70d9). The probe gate stays at 42 of 65 fixed text. 42 / 65 probes fixed text (unchanged) · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 18:54 - [F-2203] GROUNDED-DOSE arm A (SEARCHPORT on BoarLite 5b457fc7, switch on vs off): delivery holds (MAIN 26 → 40, own fact chunk 38 → 64, context 1.07×), but ungrounded dose figures rise from 5 to 9 on the 30 changed probes and 9 probes move from the fixed health text to a model answer (each with a health passage shown): FAIL; reading NOT-WORTH for the submission, switch stays off. ungrounded 9 vs 5; grounded 23 vs 8 · set 65 F3-SAFETY probes (dev) (dev, development data) · judge grounded-dose scorer (0 LLM) · n 30 changed probes of 65 · seeds 1 (42) · CI not computed · date 2026-10-24 19:13 · caveat: Quick usefulness screen skipped by a 15-minute cap. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2205] GROUNDED-DOSE arm B (X2B-BUDGET 900 on the bench engine): ungrounded 7 vs 7 (6 vs 7 once a thousands separator is parsed), grounded 23 vs 23, 0 probes leave the fixed text: PASS by the letter; but one answer took a toxic-threshold figure from a shown paracetamol passage (150 mg per kg) and gave it as the dose, grounded yet wrong, so 'grounded' is not safe by itself. ungrounded 7 vs 7; grounded 23 vs 23 · set 65 F3-SAFETY probes (dev) (dev, development data) · judge grounded-dose scorer (0 LLM) · n 65 probes (12 changed, regenerated) · seeds 1 (42) · CI not computed · date 2026-10-24 19:19 · caveat: One probe of 65, behind the model's may-be-wrong prefix; no separate noise control; adoption left to the coordinator. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2206] BoarLite ships with the Wikipedia-health limitation: the Wikipedia packs stay in the kit (dropping them cost delivered facts 42 → 33 and topics 25 → 11), and the gate counts only the first-aid, MedlinePlus and health Q&A packs as health sources, so 12 of 60 health self-tests whose article is only in a Wikipedia pack get the fixed text; a curated list of medical Wikipedia titles is a post-submission item, re-gated on the 60 self-tests and 65 probes with the grounded-dose rule plus a check that toxic thresholds are not presented as doses. ship as is; medical Wikipedia titles after the submission · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 19:32 · caveat: who: coordinator, delegated by the author; reversible: yes. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2208] SEARCHPORT Addendum 2, S2 counts: on the 51 UXMATRIX items whose kept list changed, correct-with-source 9 → 15, correct-any 20 → 21, wrong-unwarned 31 → 30 (port off → ON_NO_HEALTH) CWS 9 → 15; correct-any 20 → 21; WU 31 → 30 (of 51) (outcome-label share) · set UXMATRIX 60 (dev) (dev, development data) · judge Claude Haiku · n 51 changed items of UXMATRIX 60 · seeds 1 (42) · CI not computed · date 2026-10-24 19:35 · caveat: Metric: outcome-label counts (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio, not the ruler U-total. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. - [F-2211] BoarLite candidate contents since the first release build: health gate (B+FU detector, after retrieval, exposure rule), the fixed line 'May contain errors; check the source.' under every model answer (replacing the ported warning line), factbook v1, QuickMath, model preload, a 5-card onboarding, exposure and conversion questions answered by code with a first-aid excerpt, search port ON_NO_HEALTH; open at the build session's close: phone smoke of the final APK, APK hosting (above a 2 GB asset limit), release-key backup, About text. · date 2026-10-24 19:44 - [F-2213] After the engine used for the final ruler run (b4ef94f3), BoarLite's main branch gained two UI-only commits (brand colours always on; an animated six-card tour over the real chat UI): 10 files, 0 engine files, so the ruler run stays valid for the final engine; the release APK is rebuilt from 6aa7508a. 10 files changed, 0 under engine/ · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 20:32 · later: 2026-10-25: LabRat 7788924a changes the grounding prompt's identity line after the ruler run, so the shipped app is no longer engine-identical to b4ef94f3; the ruler number stays the number for b4ef94f3. - [F-2218] Finding from the final ruler run: BoarLite's chat shows the streamed model text, so the pipeline's cleaned answer (the citation support check CT-1 and attribution) is not used on the plain path; the run scored what the chat shows. Logged as a possible BoarLite bug. · date 2026-10-24 20:34 - [F-2219] Final BoarLite release APK (build 6aa7508a = engine b4ef94f3 plus UI-only commits: brand colours, animated six-card tour) smoke-tested on a Pixel 8a in airplane mode: fresh install, tour renders, 0 bytes of network traffic, 0 crashes, peak PSS 3.23 GB; a cited answer in 14.9 s at 16.3 tok/s and another in 9.4 s; the child-dose question gets the fixed health text; temperature conversion computed by code. PASS; 0 B network; 0 crashes; PSS 3.23 GB; 14.9 s at 16.3 tok/s · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-24 20:47 · caveat: A smoke test, not a quality measurement. One answer (minor burn) came from a first-aid source without a citation marker plus 2 unrelated sources. Health content: AI-reviewed, not clinically reviewed. · later: 2026-10-25: later APKs (cleanup ff50c5ff, LabRat 7788924a, logo 0ad476ca) replaced the 6aa7508a release APK on the phone. - [F-2220] BoarLite screens review by the author (2026-10-24 evening, UI only, engine untouched so the ruler run on b4ef94f3 still applies): a calmer first tour card, Knowledge pack icons, time and tok/s shown above the sources, no Free memory switch (always on), no PIN lock (code removed), haptics on by default, a new About page, reset texts without PIN, the chat top bar showing only the name, a drawer brand card counting the 13 knowledge packs, and the mark in the accent colour. Phone smoke of the resulting APK (b45f3aa8) on the Pixel 8a: a cited answer in 24 s, the child-ibuprofen question gets the fixed health text, 40 °C = 104 °F computed, Large text checked on all main screens. UI only; smoke on the Pixel 8a passed · set device smoke (dev) (dev, development data) · judge none (hand checks) · n hand smoke · seeds UNKNOWN · CI not computed · date 2026-10-24 23:48 · caveat: A smoke test, not a quality measurement. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2221] Long-chat summary removed from BoarLite: on the Pixel the 1.7B model used only the last 3 exchanges of a long chat, so the summary was switched off (22:10) and its code removed (owner decision, commit d14804bf) summary off; code removed · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 22:10 · caveat: Observation on one phone, not a measured arm. - [F-2222] Product cleanup round on BoarLite (2026-10-24 22:55 → 2026-10-25 01:11, owner decisions): the Expo/React Native tree removed (the app ships the native Android code only); code unreachable in the closed app removed (Deep Research beyond the recap, the Detailed style, persona and length options, adaptive routing, the long-chat summary, PIN and sealed-chat legacy); Portuguese support removed (English only, except words that are also English); Settings without the Knowledge section or the title switch (titles always on), Clear chat history moved under Start over, 50 chats kept instead of 10; MIT licence texts shipped in About; docs describe the closed app. Engine unit tests: 274, all passing; each step device-checked on the Pixel 8a. main ff50c5ff; 274 engine tests · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 01:11 · caveat: Product changes; no ruler rerun after them (see the persona-line row). - [F-2223] Between the ruler engine b4ef94f3 and LabRat 7788924a the engine module changed in 124 files (643 lines added, 97,537 removed), almost all removals of code and test resources described as unreachable in the closed app; the frozen DEVICE-FINAL criterion describes 7788924a's engine as b4ef94f3 plus the Rat persona line. Behaviour checks after the cleanup: 274 engine unit tests and a 5-question device smoke whose retrieved titles, health gate and conversion match the baseline; no ruler rerun. 124 files; +643 / −97,537 lines · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: Line counts from a read-only git diff of the engine module; the smoke compares titles, gate and conversion, not answer quality. - [F-2226] LabRat logo n8 chosen by the author after 4 logo rounds: a magnifier whose ring is a rat's head and whose handle is its tail (commit 0ad476ca); its APK installed on the Pixel 8a, smoke 6/6 matching the baseline, app screen and launcher icon checked on the phone. logo n8; smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 11:24 · caveat: A smoke test, not a quality measurement. - [F-2228] DEVICE-FINAL started overnight on 2026-10-25 at the author's request (final benchmarks with the app running the engine on the phone, for publication): the device eval runner was patched on an eval branch to answer follow-up turns with the chat's history and to record the streamed text the chat shows; engine logic untouched; the unbundled release was installed over the bundled data with the same signing identity; a pre-run smoke and a follow-up-turn check passed. running · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. - [F-2229] An APK install on the Pixel 8a interrupted the DEVICE-FINAL run on 2026-10-25 11:28; the run was restarted, and the phone is kept free of installs until the matrix log ends. · date 2026-10-25 11:28 · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. Process fact only. · later: 2026-10-25 final: the run was resumed (69 complete items kept, 6 re-run), not restarted from scratch. - [F-2230] APK hosting decided: the 3.04 GB LabRat APK goes to Hugging Face (author's OK on 2026-10-25) because GitHub release assets must be under 2 GiB; a public install page is planned with the download link, sha256, about 6 GB free space, Android 8+, "install unknown apps", the Play Protect prompt and a first launch of about 12 s. The code repository stays private unless the author decides otherwise. Hugging Face; upload pending · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: As of this update the upload is not recorded as done. · later: 2026-10-25 15:3x: the APK is hosted on Hugging Face (huggingface.co/r4topunk/labrat); the code repository is public (labrat-boar-fork). - [F-2231] Install-path limits found before release: Android developer verification is live since 2026-09-30 in Brazil, Indonesia, Singapore and Thailand; adb installs are exempt, but browser installs of apps from unregistered developers may hit an "advanced flow" (a warning and a one-day wait), scope unconfirmed. The test phone runs GrapheneOS, which is not Google-certified, so it cannot prove the judges' install path; a browser-install test on a stock certified phone is planned, and registering the app with the developer verification is the mitigation (needs the author's Google account) risk open · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: Policy scope as checked by the build session on 2026-10-25, not confirmed on a certified phone. - [F-2232] Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat shows. open; fix needs a ruler rerun · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: No fix is in the record as of this update. - [F-2233] Claim-text plan (2026-10-24 PM hand-off): lead with what the app does and the final ruler numbers labelled with both judge families, the seed count, "desktop twin" and the win-rate, plus measured product numbers each labelled with the engine they come from; wording goes to an independent reviewer, then the author. Also pending: a decisions-and-evidence page for the judges, a short demo post and the app's About text. plan; not posted · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 20:5x · caveat: The ruler numbers stay internal until the author rules on AD-15; this row states the plan, not the numbers. - [F-2234] Post-submission backlog (not before the submission): medical Wikipedia titles as health sources; a dose guard (cut ungrounded or toxic-threshold doses) so the search port can run on health turns; X2B-BUDGET 900 and RETBENCH k4+comp40 (both passed the grounded-dose re-gate) into the native search port; factbook v2 with the new search. · date 2026-10-24 20:5x · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. - [F-2243] DEVICE-FINAL M4, the 60 UXMATRIX items on the phone: correct-any 0.350 (0.233–0.467), wrong-any 0.617. correct 0.35 of 60 (outcome-label share) · set UXMATRIX 60 (dev) (dev, development data) · judge Claude Haiku 5.5-low labels (UXMATRIX protocol) · n 60 · seeds 42 · CI 0.233-0.467 · date 2026-10-25 13:2x · caveat: Outcome-label shares, not the ruler scale. Correct-with-source is a lower bound (phone rows hold source titles only). - [F-2244] DEVICE-FINAL ran as one interrupted and resumed run: seed 42 stopped at 77 of 88 turns at 11:31 when an APK install of LabRat 0ad476ca (brand files only) reached the phone; the 69 complete items were kept and the 6 unfinished ones were re-run on 0ad476ca; seeds 43 and the M2-M5 matrix ran entirely on 0ad476ca; seed 44 was dropped for time. interrupted once, resumed; seed 44 dropped · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 11:31 · caveat: Process fact. - [F-2251] Submission post prepared: one post with a 60 s video and a short poidh text, numbers from DEVICE-FINAL; as of the last record row it was not posted. prepared; not posted · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 16:29 - [F-2252] Article "RatLab — A Boar Research" published on GitHub Pages at r4topunk.github.io/boar-research (16:33), updated at 16:46 with the DEVICE-FINAL result and links to the public repositories. article live · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 16:46 ## Part 9 · Limits, licence and citation - Research record 1.0 as of 2026-10-25: results are the standing on that date, not a final result in any wider sense; the open fronts say what could still change it. Check https://r4topunk.github.io/boar-research/data/status.json for anything newer. - Three metric families never share an axis: the judge.v1 ratio, outcome-label shares and the ruler U-total. - Text and data CC BY 4.0; code Apache-2.0; third-party material keeps its own licence. - Cite as: r4to, independent researcher. RatLab — A Boar Research (2026), research record 1.0, as of 2026-10-25. https://r4topunk.github.io/boar-research/cite/ · https://r4topunk.github.io/boar-research/CITATION.cff. - AI-reviewed, not clinically reviewed. Nothing here is medical advice.