# 12 · A second ruler (bench v0.4) > Closed record · research record 1.0 · as of 2026-10-25 16:46; the author closed the research lines on 2026-10-23. Current standing, not a final result: Open fronts: https://r4topunk.github.io/boar-research/data/status.json. > > Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar - Page: https://r4topunk.github.io/boar-research/research/new-ruler/ · Markdown twin: https://r4topunk.github.io/boar-research/research/new-ruler.md · Whole paper: https://r4topunk.github.io/boar-research/llms-full.txt · Index: https://r4topunk.github.io/boar-research/llms.txt - Span: 2026-10-19 → 10-24 **In one sentence.** The first ruler scored a desktop engine against Claude-written references with a Claude judge on Claude-written questions. For the submitted app the project wrote a second one: 75 new questions frozen on 2026-10-20 before any system answered them, written half by each judge family, scored as U-total with a zero for non-answers, two judge families and a split by whether the packs hold the answer. As of 2026-10-25 it has references from both families, a desktop-twin run of the submitted app's pre-rename engine that the author published as history (0 of 71 wins; lower family below the 0.50 bar), and an on-phone run in progress that will carry the headline. ## For curious readers {#curious} #### Our first exam graded the wrong thing The first ruler compared our desktop engine with answers written by one AI family, on questions written by that family, graded by that family. The bounty asks about an app on a phone. So we wrote a second ruler. #### 75 new questions, sealed before anyone answered 75 new questions, written half by Claude and half by an OpenAI model, fact-checked by both, and frozen on 20 October before any system had seen them. It is not written outside the Claude family; we say so. *Badge:* bench v0.4 · 75 questions (71 scored + 4 anchors) · frozen 2026-10-20. #### A blank answer now scores zero On the old scale a polite refusal still earned a few points. The new ruler first asks "did this even try to answer?"; if not, it scores zero. Then both AI families grade it, and the lower grade counts. #### Graded, not yet published By the evening of 24 October both AI families had written their answer keys, and the submitted app's engine had sat the exam on a desktop copy of the phone app, graded by both families. The grade exists; whether it is published is the author's call, and until then this page shows no number and no direction. A test on the phone itself started on 25 October; it has no result yet. **In one sentence card:** "A fairer exam is written and sealed; the submitted app has sat it, and the grade waits for the author (as of 24 October 2026, 20:34)." *Facts: [D-131] [F-2070] [F-1416] [F-2081] [F-1529] [G-u-total] [F-2110] [F-2212] [F-2214]* ## For tech people {#tech} ### 75 items frozen before any answer, U-total with a stage-1 zero, two judge families; on the desktop the submitted engine stayed below the bar, the phone run is the headline Bench v0.4 has 75 items (71 headline + 4 anchors taken from the example questions in Vitalik Buterin's post that inspired the bounty, scored outside the headline), frozen on 2026-10-20 at 16:45 (sha f9ad2f2f) before any system answered; candidates came from Claude (36) and OpenAI gpt-5.6-sol (34) and were cross-fact-checked (329 checklist points supported on the open web). The score is U-total: Σ(app rubric mean − 1) / Σ(reference rubric mean − 1) over the 71 headline items, with health items weighted 0.5 (weight-1 beside), so the rubric floor maps to 0. Because judge.v1 still gave a bare refusal U 0.058, a stage-1 classifier (DECLINE2STAGE) zeroes detected non-answers before judge.v1 scores the rest (RULING R1). The verdict arm installs all six persona packs at once (A∪); the verdict statistic is the lower of the two judge families per bootstrap draw, and "meets the bar" requires CI low > 0.50 with every gate (RAM, storage, offline, speed, safety) passed. Spec r3 and r4 were frozen on 2026-10-21 before any app answer; r4 adds an information-available split: the packs hold the answer for 19 of 71 headline items, partly for 25, not at all for 27. As of 2026-10-24 20:34 both reference families are complete (75/75 each; the gpt-5.6-sol half at 20:05, 0 hard failures; [F-2212]), and three engines have answered on desktop twins. The last of them is the final run on BoarLite's engine b4ef94f3 (RULER-FINAL): one seed instead of the spec's two (the author's choice), each judge family against its own references, stage 1 not run, no errata, no phone run [F-2214] [D-204]. Published by the author's decision of 2026-10-25 as history: 0 of 71 wins under both judge families (one gpt-5.6-sol tie); U-total (desktop twin of engine b4ef94f3) Claude Opus 0.164 (0.123–0.208), OpenAI gpt-5.6-sol 0.245 (0.194–0.300); the lower family, 0.164 (0.123–0.206), is below the 0.50 bar; by information available (Claude family) 0.164 where the packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27): with the information present, the 1.7B did not deliver it [F-2215] [F-2217]. The paired delta against the native engine branch stays internal (it prints that branch's number). They are numbers for b4ef94f3: the shipped LabRat 7788924a changes the grounding prompt's identity line after this run [F-2225], and the on-phone re-measure is DEVICE-FINAL (OF-8, running; criterion frozen 2026-10-25, [F-2227]) (the ruler run predates the Rat persona line). **Rule box:** "No item text, reference or label list is published before the result card" [D-195]. "Any engine change after 2026-10-24 counts as developed after exposure" [F-2108]; "v0.4 is the claim set: no tuning after the final run" [D-204]. ## For AI professionals {#ai} ### The record for this chapter **Key facts:** - Design (10-19): a public ruler with a true-zero utility, uncapped two-family web references, items written outside the judge family where possible, and the submitted app as the scored system ([D-131] / [F-1383]). - Bench v0.4: 75 items (71 headline + 4 anchors taken from the example questions in Vitalik Buterin's post that inspired the bounty, scored outside the headline), frozen 2026-10-20 16:45, sha f9ad2f2f, before any system answered [F-2070] [F-1353]. Candidate writers: Claude 36, OpenAI gpt-5.6-sol 34 [F-1416]. 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both). It is **not** written outside the Claude family: half is Claude-written, a stated deviation [G-ruler-v04]. Its item text and references are not public [D-195]. - Metric: U-total, a ratio of sums of (rubric mean − 1) for the app over the reference, so the rubric floor maps to 0; health items weigh 0.5 with weight-1 printed beside ([D-137]; the weighting was decided after the items existed, [F-1389]); verdict = the lower of the two families, "meets the bar" only if CI low > 0.50 with all gates passed [G-u-total]. - Why a stage-1 zero: judge.v1 gives a bare refusal U 0.058 (0.037–0.081), a hedge 0.053, a fluent no-content answer 0.016 (Haiku, 30 v0.1 dev ids; [F-2081]). A binary-checklist judge v2 failed its retest gate (CE retest 0.966 vs a needed 0.974; [F-1431]). RULING R1 kept judge.v1 unchanged and added DECLINE2STAGE, a classifier that zeroes detected non-answers; it was a coordinator override of a frozen FAIL, accepted with conditions by the spec reviewer ([D-144] / [F-1529]). - Spec r3 (sha 0fac2286, 20:48) and r4 (sha 82772e33, 21:18) were frozen on 2026-10-21 before any app answered a v0.4 item [F-2109]. Verdict arm A∪: the app with all six persona packs installed at once [D-139] [D-156]. - References: Claude half 75/75 frozen before any app answer (uncapped, median 703 words, median 6 sources) [F-2110]; OpenAI gpt-5.6-sol half 75/75 on 2026-10-24 20:05, 0 hard failures [F-2212]. - Information available (counts only; label lists quarantined): the A∪ packs hold the answer for 19 of 71 headline items, partly for 25, not at all for 27 (strict 15 / 23 / 33) [F-2111]. 7 current-city items carry 10.7 % of headline weight and no app could answer them [F-2113] [F-1960]. - Exposure: on 2026-10-24 two engines answered v0.4 for the first time (desktop twins, Claude half only, stage 1 not run). Those numbers stay internal (one of the two engines is the native engine branch, whose numbers stay with the app maintainer; AD-15 (b)) and are dev/parity evidence, never a verdict; any later engine change counts as developed after exposure [F-2108]. - The scored system is the submitted APK on a phone [D-151] [D-191]. As of 2026-10-24 20:34 no phone run on v0.4 exists. - Final run (RULER-FINAL, 2026-10-24 20:34): BoarLite's final engine b4ef94f3 on its desktop twin, 71 headline items, one seed (the spec asks for two; the author's choice), each judge family against its own references, stage 1 not run [F-2214] [D-204]. The ruler is now exposed to three engines. Published by the author's decision of 2026-10-25 as history: 0 of 71 wins under both families (one gpt-5.6-sol tie); U-total (desktop twin of engine b4ef94f3) Claude Opus 0.164 (0.123–0.208), OpenAI gpt-5.6-sol 0.245 (0.194–0.300); the lower family, 0.164 (0.123–0.206), is below the 0.50 bar [F-2215] [X-405]. Information-available split, Claude family: 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27): with the information present, the 1.7B did not deliver it [F-2217]. The paired delta against the native engine branch stays internal, because it prints that branch's number (U44). They are numbers for b4ef94f3: the shipped app, LabRat 7788924a, adds the Rat persona line to the grounding prompt after this run [F-2225]. The on-phone re-run of v0.4 on LabRat (DEVICE-FINAL, three seeds under Claude Opus, seed 42 under gpt-5.6-sol) started on 2026-10-25 under a criterion frozen before generation ([F-2227]; OF-8); no result is in this record. **Key decisions:** [D-131], [D-132], [D-133] (freeze; one item dropped after both families contradicted it), [D-137], [D-138], [D-139], [D-144] (RULING R1), [D-151], [D-156], [D-157], [D-158], [D-195]. **Supersessions to show:** the v0.3 path to an unexposed number (D5, 2026-10-18, [F-1374]) → v0.4, which is exposed and half Claude-written; "the ruler was frozen on 10-20" → the *item set* was frozen on 10-20, the first frozen spec is r3 on 10-21 [F-1628]. **Turning point:** 2026-10-20 16:45: the v0.4 item set is frozen before any system has seen it. ### 3.6 A second instrument for the submitted app Because the first instrument scored a desktop engine against Claude-written references on Claude-written items, a second ruler was specified for the submitted app [D-131]. Bench v0.4 has 75 items (71 headline; 4 anchors taken from the example questions in Vitalik Buterin's post that inspired the bounty, outside the headline), drafted by Claude (36 candidates) and OpenAI gpt-5.6-sol (34), cross-fact-checked (329 checklist points supported on the open web by at least one family with none contradicting), and frozen on 2026-10-20 before any system answered (sha f9ad2f2f) [F-2070] [F-1416]. The headline statistic U-total is a ratio of sums of rubric mass above the floor (mean − 1) against uncapped frontier-plus-web references from both families, with health items at weight 0.5 (weight-1 beside; decided after the items existed, [F-1389]) and a stage-1 classifier that zeroes detected non-answers (DECLINE2STAGE), because judge.v1 scores a bare refusal at U 0.058 (0.037–0.081) [F-2081]. A binary-checklist judge v2 failed its pre-set retest gate [F-1431]; keeping judge.v1 with the stage-1 zero (RULING R1) overrode a frozen FAIL and was accepted with conditions by the spec reviewer [D-144]. Spec r3 (sha 0fac2286) and r4 (sha 82772e33, adding an information-available split, FACT-AVAIL, and ORACLE / ORACLE-0 rows) were frozen on 2026-10-21 before any app answer [F-2109]. The verdict arm installs all six persona packs (A∪); the verdict is the lower family per bootstrap draw with CI low > 0.50 and all gates. FACT-AVAIL labels: AVAILABLE 19 / PARTIAL 25 / NONE 27 of 71 (strict 15 / 23 / 33), lists quarantined [F-2111]. As of 2026-10-24 20:34: references 75/75 for both families (Claude median 703 words; gpt-5.6-sol completed 20:05, 0 hard failures, [F-2212]); the set counts as exposed from 2026-10-24 [F-2108] and has now been answered by three engines on desktop twins. The final run (RULER-FINAL, [F-2214], [D-204]) scored BoarLite's engine b4ef94f3 with one seed (spec: two), each judge family against its own references (not the two-family mean), stage 1 not run, no errata; the spec labels desktop-twin numbers dev/parity evidence. Published by the author on 2026-10-25 as history: win-rate 0 of 71 under both families (one gpt-5.6-sol tie); U-total (desktop twin of engine b4ef94f3) 0.164 (0.123–0.208) Claude Opus, 0.245 (0.194–0.300) gpt-5.6-sol; minimum over families 0.164 (0.123–0.206), below the bar by the pre-registered rule; FACT-AVAIL (Claude) AVAILABLE 0.164 (n 19), PARTIAL 0.174 (n 25), NONE 0.155 (n 27), so the information's presence did not raise U-total [F-2215] [F-2217]. The paired delta against the native engine branch is withheld (it carries that branch's absolute; the app maintainer's numbers stay internal). The engine it measured, b4ef94f3, precedes the rename to LabRat: 7788924a changes the grounding prompt's identity line ("You are Rat…"), so the RULER-FINAL number is not attributed to LabRat without that caveat [F-2225] (the ruler run predates the Rat persona line). DEVICE-FINAL (criterion frozen 2026-10-25 03:3x, [F-2227]) re-runs v0.4 on the phone with LabRat 7788924a, three seeds under Claude Opus and seed 42 under gpt-5.6-sol, with a phone-vs-desktop parity read against b4ef94f3 that is not a verdict; running, no results in this record. ## Facts cited {#facts} Resolve every [F-…] marker in https://r4topunk.github.io/boar-research/data/facts.json. Each line gives the claim, the number with its metric family, set and status, judge, n, seeds, CI, date and caveat. - [F-1353] v0.4 ruler item set FROZEN (75 items, sha f9ad2f2f...) before any system answered. · date 2026-10-20 - [F-1374] D5: no bench v0.3 (non-Claude held-out set) before closure. record declares no unexposed set; later partly superseded by v0.4 ruler (see §6) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-18 · caveat: who decided: delegated coordinator; why: ~14-24 h of author item writing; would land after the deadline; bar needs ~0.53 at n 110 vs best ~0.43-0.51; serves honesty, not winning; alternatives: 60-item wave 1; 6-item pilot; reversible: yes | VERIFIER NOTE: Base OF-3 says 'the author decided'; precise form: the author delegated D1-D7 and the coordinator's recommendation (skip v0.3) was adopted ( l.4). - [F-1383] Build a public ruler: true-zero utility U + task success, uncapped two-family web references, non-Claude items, submitted app on phone, gates, speed, install. led to v0.4 items + spec v1 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-19 · caveat: who decided: coordinator from 8-agent workflow (both critics converged); why: five misreadings of the bounty in the old bench; alternatives: keep old bench as headline; reversible: yes - [F-1389] Ruler headline weights the 11 health_safety items at 0.5, adds a safety gate, publishes weight-1 U beside it. in spec v1 §2; reviewer F3 flags timing (decided after items existed) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: who decided: author; why: bounty asks for casual lookup/research, never names health; alternatives: weight 1; reversible: yes AI-reviewed, not clinically reviewed. - [F-1416] v0.4 candidate writers. Claude 36, OpenAI gpt-5.6-sol 34; 302/307 points supported, 5 disputed · set v0.4 drafts (frozen ruler set (held-out for the ruler; counts only)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 10-20 - [F-1431] Judge v2 (binary checklist) test-retest. CE test-retest 0.966 vs v1 0.924 (needed 0.974); SD 0.80x (needed 0.60x) -> NO-GO · set dev (dev, development data) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI frozen rule · date 10-20 - [F-1529] RULING R1: judge = judge.v1 unchanged + DECLINE2STAGE zero per detected non-answer; D 0.058 kept as sensitivity row. adopted into ruler r3/r4 (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: delegated coordinator (override of a frozen FAIL, accepted with conditions by reviewer); why: judge v1.1/v1.1b changed CE verdicts (5-7/40 flips); DECLINE2STAGE's 2 bar-(b) hits were engine refusal strings; alternatives: judge v1.1; decline subtraction only; reversible: yes (pre-exposure) - [F-1628] Earlier: "the ruler was frozen on 2026-10-20". Now: on 2026-10-20 the v0.4 item set was frozen; spec r2 was a draft (its binary-checklist judge v2 was NO-GO); RULER-SPEC v1 r3 (2026-10-21) is the first frozen spec, r4 adds the information-available split. Changed to: Ruler = RULER-SPEC v1 r3/r4: judge.v1 + DECLINE2STAGE, two families, A-union verdict arm, FACT-AVAIL split (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: Verification note: Earlier standing is from slice u-1820, not the base: on 10-20 the v0.4 ITEM SET was frozen (not 'the ruler'); spec v1 r2 (binary checklist judge) was a draft, judge v2 already NO-GO . r3 is the first frozen spec. - [F-1960] RV4-CITY-BOUND: How much can the 7 current-city items swing v0.4? 7 items = 10.7 % of headline weight; swing ≥ 0.107 U · set v0.4 tags (held-out, script) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge arithmetic · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: criterion frozen: yes (53f75edc); arm verdict: KILL branch: separate stratum scored as given - [F-2070] Bench v0.4 frozen. 75 items (71 headline + 4 anchors); 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both families); sha f9ad2f2f (ruler U-total) · set v0.4 (held-out ruler) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: written by Claude + OpenAI gpt-5.6-sol (the judge families) - [F-2081] judge.v1 does not give declines a true zero. bare refusal U 0.058 (0.037-0.081); hedge 0.053; fluent no-content 0.016 (judge.v1 ratio) · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 30 v0.1 ids · date 2026-10-21 · caveat: leak in correctness dimension - [F-2108] Bench v0.4 exposure. first answers by two engines 10-24; any later engine change = developed after exposure (ruler U-total) · set v0.4 (held-out ruler, now exposed) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Opus · n UNKNOWN · seeds UNKNOWN · CI 75 items × 2 seeds × 2 engines · date 2026-10-24 · later: 2026-10-24 20:34: a third engine (BoarLite b4ef94f3) answered v0.4 in the final ruler run; its numbers are internal pending the author. - [F-2109] Ruler spec frozen. r3 sha 0fac2286 (10-21 20:48); r4 sha 82772e33 (21:18) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: private pre-registration - [F-2110] Claude references for v0.4 frozen before any app answer. 75/75 items, 88 turns, uncapped, median 703 words, median 6 sources · set v0.4 (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Opus + web · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: OpenAI gpt-5.6-sol half 15/75 by 10-24 - [F-2111] Information-available labels on v0.4. AVAILABLE 19 / PARTIAL 25 / NONE 27 (strict 15/23/33) of 71 · set v0.4 (quarantined labels) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Haiku + Claude Opus · n UNKNOWN · seeds UNKNOWN · CI validation 34/40 · date 2026-10-21 · caveat: counts only; lists stay quarantined - [F-2113] Current-city stratum weight. 7 items = 10.7 % of headline weight (7/65.5) · set v0.4 tags (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge arithmetic · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 - [F-2212] REFS-V2, OpenAI gpt-5.6-sol half complete: 75 of 75 v0.4 references written with live web search under the frozen protocol, 0 hard failures, 97 calls (two items completed on a resumed run); references of both judge families now exist. 75 / 75; 0 hard failures · set v0.4 (held-out ruler) (held-out ruler (exposed)) · judge UNKNOWN · n 75 items (88 turns) · seeds UNKNOWN · CI not computed · date 2026-10-24 20:05 · caveat: Protocol frozen 2026-10-21 before any call (prompt 5e41177d, generator eb2fd7e2). Reference text not read by the coordinator. - [F-2214] Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as F-2215 and F-2217; history, the headline is DEVICE-FINAL) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; now exposed to three engines)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · date 2026-10-24 20:34 · caveat: Deviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run. - [F-2215] RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 items. Claude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0 (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · n 71 headline items · seeds 1 (42) · CI Claude 0.123-0.208; gpt-5.6-sol 0.194-0.300; min 0.123-0.206 · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8). - [F-2217] RULER-FINAL information-available split: Claude family U-total 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27); gpt-5.6-sol family 0.272 / 0.238 / 0.233: with the information present, the 1.7B does not deliver it. AVAILABLE 0.164 vs NONE 0.155 (Claude) (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol · n 19 / 25 / 27 headline items · seeds 1 (42) · CI not computed · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Pre-registered stratifier (spec r4). Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8). - [F-2225] Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phone. persona line changed after RULER-FINAL · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a. - [F-2227] DEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine change. matrix M1-M5; claim numbers = M1 + M5 (ruler U-total) · set bench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60 (dev, development data) · judge Claude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5) · n 75 items × 3 seeds + 265 product items · seeds 42, 43, 44 (M1); 42 (M2-M4) · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update. ### Decisions cited {#decisions} - [D-131] 2026-10-19 · Build a public ruler: true-zero utility U + task success, uncapped two-family web references, non-Claude items, submitted app on phone, gates, speed, install · why: five misreadings of the bounty in the old bench. · status: kept - [D-132] 2026-10-20 · Item families from Vitalik Buterin's own public test questions; travel ~1/3 not >= 50 %; 8 multi-turn; over-refusal probes; venue "best" checked on open web · why: match the inspiring post's queries. · status: kept - [D-133] 2026-10-20 · Resolve 5 disputed points; drop one item contradicted by both families; tag 4 V-exact anchors out of the headline; freeze v0.4 · why: both-family fact check. · status: kept - [D-137] 2026-10-20 · Ruler headline weights the 11 health_safety items at 0.5, adds a safety gate, publishes weight-1 U beside it · why: bounty asks for casual lookup/research, never names health. · status: kept - [D-138] 2026-10-20 · Ruler headline = coverage x U decomposition + U-total; product refuses by default when no source is kept, with a "show anyway" control · why: separate what packs answer from memory answers. · status: kept - [D-139] 2026-10-20 · Verdict arm = A∪ (all persona packs installed once, same for every item); per-family arm A kept as a labelled row · why: per-family arm = oracle topic routing. · status: kept - [D-144] 2026-10-21 · RULING R1: judge = judge.v1 unchanged + DECLINE2STAGE zero per detected non-answer; D 0.058 kept as sensitivity row · why: judge v1.1/v1.1b changed CE verdicts (5-7/40 flips); DECLINE2STAGE's 2 bar-(b) hits were engine refusal strings. · status: kept - [D-151] 2026-10-21 · Final score = submitted native APK on Pixel 8a, v0.4 frozen (75 items), Claude + OpenAI gpt-5.6-sol judges, frontier + web references · why: the bounty judges the app. · status: kept - [D-156] 2026-10-21 · Verdict arm = app with packs (A-union); OpenAI gpt-5.6-sol as second judge family; headline waits for new RC; ruler in its own session · why: thesis "app + packs answers well"; two families. · status: kept - [D-157] 2026-10-21 · Freeze RULER-SPEC v1 r3 (private) · why: pre-registration before any v0.4 answer. · status: kept - [D-158] 2026-10-21 · Freeze r4: add information-available split (FACT-AVAIL) + ORACLE/ORACLE-0 rows; U-total stays verdict · why: separate "packs lack it" from "app fails to deliver it" · status: kept - [D-191] 2026-10-21 · Honest 8 GB UX; no 35B deep mode delivered; final score = submitted native APK on Pixel 8a with v0.4, Claude + OpenAI gpt-5.6-sol judges · why: deep tier needs 12 GB and integration work. · status: kept - [D-195] 2026-10-21 · No public upload of v0.4 items or references; labels of FACT-AVAIL quarantined (coordinator-only) until the result card · why: held-out discipline. · status: kept - [D-204] 2026-10-24 · Final ruler run frozen: BoarLite b4ef94f3, v0.4, one seed, both judge families — decision: Run v0.4 (spec r4) on the desktop twin of b4ef94f3 with one seed (42), Claude Opus vs Claude references and gpt-5.6-sol vs its references; no tuning after this run; engine frozen unless the phone smoke forces an engine fix. · why: one seed was the author's choice; stated in every number. · status: kept ### Experiments cited {#experiments} - [X-405] RULER-FINAL · Ruler v0.4 on BoarLite's final engine (desktop twin) · question: What is BoarLite's U-total on v0.4 under both judge families? · result: U-total Claude 0.164 (0.123-0.208), OpenAI gpt-5.6-sol 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 under both (1 gpt-5.6-sol tie) · set v0.4 (71 headline + 4 anchors), judge Claude Opus + OpenAI gpt-5.6-sol, n 71, seeds 1 (42), CI Claude 0.123-0.208; gpt-5.6-sol 0.194-0.300; minimum 0.123-0.206 · verdict: measurement (run complete; below the 0.50 bar on the lower judge family; published 2026-10-25 as history (desktop twin of b4ef94f3, before the Rat persona line); the headline is DEVICE-FINAL (OF-8)) --- Cite as: r4to, independent researcher. RatLab — A Boar Research (2026), research record 1.0, as of 2026-10-25. Text and data CC BY 4.0. Fact ids or a link must travel with a quote. How to cite: https://r4topunk.github.io/boar-research/cite/ · https://r4topunk.github.io/boar-research/CITATION.cff. AI-reviewed, not clinically reviewed. Nothing here is medical advice.