RatLabA Boar Research
Closed recordas of what’s still open →

The research · Act V, Where it stands · Chapter 17

Open fronts and hand-off

As of 2026-10-25 03:30 four fronts are open: the submitted app's delivery and scoring (“Scoring of the submitted app (LabRat, formerly BoarLite)”, LabRat, formerly BoarLite: final bundled build, Hugging Face upload and install page, the install-path check, its scores now public by the author's decision of 2026-10-25), the final on-phone matrix (the final on-phone run, running), the claim text (“Claim text on the bounty”) and the shaped-pack publication (“Shaped factbook and pack publication”); the ruler run (“Ruler v0.4 verdict run”) landed on the desktop run of the earlier engine and is published as history.

Full thesis

Three earlier fronts were dropped, each by a dated decision: the deep tier on a phone (“Deep tier on a 12 GB-class phone”) and expert streaming on Android (“Expert-streaming engine on Android”) when the author ruled out a 35B mode (2026-10-21), and v0.3 (“Held-out set written outside the Claude family (bench v0.3)”) when the delegated decisions chose no v0.3 before closure (2026-10-18). Each front lists what would change, in both directions; criteria are frozen in the research repository before each run. The deliverables already public stay usable by other teams.

2026-10-14 →

Everything is public

The first exam, the judge and the research record are on GitHub; the libraries are on Hugging Face. Anyone can run their own assistant against the same answer key.

What is still being finished

Finishing LabRat (the final build and its upload to Hugging Face); the final test of the app on the phone, whose grade is now the headline; posting the revised claim with a link to this article. The reshaped factbook library stays unpublished. Each card says what changes if it works and what changes if it doesn't.

What we stopped testing, and why

The big model on a phone (dropped when the author ruled out a deep mode), streaming it from storage on Android (built, never measured), and a sealed exam written entirely outside the Claude family (never written).

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

The numbers that drive this chapter

  • Public repos live 2026-10-15 10:07: boar-bench (139 files) and boar-research (411 files); scrubbed, mechanically gated, independently reviewed (41/44 findings applied) Public repos at releaseboar-bench 139 files / 2.26 MB; boar-research 411 files / 5.53 MBDate2026-10-15 01:18 / 10:07Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatone commit each, author "BOAR R&D"Sourcenot in the public repositories yetOpen the data file.
  • Engine pull request to the upstream app opened 10-15 10:10 PR #85 contents at opening13 cut 7 commits rebased + 1 test + 2 wiring commits; vitest 1,378/1,378; replay 60/60; generation 59/60Setv0.1 publicJudgenoneDate2026-10-15 10:10Kinda record, not a measurement (status, n, seeds: not recorded)Caveatlater + CodeRabbit fixes, NUMGUARD r3, HEALTHGAP r2 (tip 8ac9eb7a)Sourceresearch record arms/pr1/PR1-REBASE.mdOpen the data file; later a number guard and a health-gap fix added.
  • Pack uploads after 10-17: a fixed Simple English Wikipedia revision (a unit-conversion converter bug, 3,176 → 524 artefact chunks) on 10-21 and a knowledge-base index snapshot on 10-22 Public pack uploads in slicesimplewiki r2 (convert fix) 10-21; kb-index-research snapshot 10-22Statusn/aDate10-21/22Kinda record, not a measurement (set, judge, n, seeds: not recorded)Caveatstanding pack-workflow OKSourcenot in the public repositories yetOpen the data file. The shaped factbook is drafted as user-selected only and unpublished (“Shaped factbook and pack publication”).
  • A reusable 0-LLM retrieval bench (retrieval bench) exists in the private research log; its publication is not recorded as of 2026-10-24 RETBENCH best (k4 + compress 0.4): MAIN 42→45closed by probes (2 dose/number changed)SetdevStatusdevJudge0-LLMn97Seedsnot recordedDate2026-10-23CaveatRETBENCH v1 = reusable 0-LLM bench AI-reviewed, not clinically reviewed.Sourcenot in the public repositories yetOpen the data file.
  • A hand-off package for the app maintainer is prepared; sending it waits for the author. Citations.applyShown stays a documented known limitation by the author's decision of 2026-10-25 Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat showsopen; fix needs a ruler rerunDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatNo fix is in the record as of this update.Sourcenot in the public repositories yetOpen the data file.
  • Power for any future bar test: at n = 110 a one-sided "> 0.50" needs a point estimate ≈ 0.53 (≈ 0.545 with one-seed run noise) Bar test power at n = 110: required point estimate ≈ 0.53 without run noise, ≈ 0.545 with one-seed run noise; a true 0.53 passes only ≈ 29-50 % of the time.0.53 / 0.545Setv0.3 plann110Date2026-10-12Kinda record, not a measurement (status, judge, seeds: not recorded)Caveatt-based, not bootstrapSourcebenchmark bench/v0.3/plan/power-statement.md:14-18Open the data file; v0.4's 71 headline items are fewer.
  • Dropped: (deep tier on a 12 GB-class phone), (expert streaming on Android: built 10-19, never measured on a phone) Expert-streaming engine cross-built for Androidbuilds; phone numbers unmeasuredDate2026-10-19Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatno phone run in this sliceSourcenot in the public repositories yetOpen the data file, (v0.3: no items; its role passed to v0.4, which is half Claude-written) D5: no bench v0.3 (non-Claude held-out set) before closurerecord declares no unexposed set; later partly superseded by v0.4 ruler (see §6)Date2026-10-18Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: delegated coordinator; why: ~14-24 h of author item writing; would land after the deadline; bar needs ~0.53 at n 110 vs best ~0.43-0.51; serves honesty, not winning; alternatives: 60-item wave 1; 6-item pilot; reversible: yesVERIFIER NOTE: Base OF-3 says 'the author decided'; precise form: the author delegated D1-D7 and the coordinator's recommendation (skip v0.3) was adopted ( l.4).Sourcenot in the public repositories yetOpen the data file.

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

§11 Limitations and open fronts

As of 2026-10-25 the record contains no unexposed held-out number and no number written entirely outside the Claude family: v0.3 was dropped (2026-10-18), and the second ruler is half Claude-written and was exposed on 2026-10-24. Its final run on BoarLite's engine (“Ruler v0.4 verdict run”, landed 2026-10-24 20:34) is a desktop twin with one seed, published as history (lower family 0.164, below the bar); it measured b4ef94f3, before the Rat persona line of LabRat 7788924a; the on-phone run of LabRat (DEVICE-FINAL, “DEVICE-FINAL: final on-phone matrix”) landed on 2026-10-25 (the headline); with 71 headline items its power is below that of the 110-item design (at n = 110 a one-sided "> 0.50" needs a point estimate near 0.53) Bar test power at n = 110: required point estimate ≈ 0.53 without run noise, ≈ 0.545 with one-seed run noise; a true 0.53 passes only ≈ 29-50 % of the time.0.53 / 0.545Setv0.3 plann110Date2026-10-12Kinda record, not a measurement (status, judge, seeds: not recorded)Caveatt-based, not bootstrapSourcebenchmark bench/v0.3/plan/power-statement.md:14-18Open the data file. The deep tier never ran on a phone and was not delivered (“Deep tier on a 12 GB-class phone” and “Expert-streaming engine on Android” dropped). The thesis's evidence rests on pack-written self-tests labelled by one cheap judge with one seed. The human label check was not done. BoarLite's health rules are fitted on the 65 probes they are checked on, and its search port was adopted against a failed frozen safety verdict (§6.11). Open fronts (each with what would change in either direction; criteria frozen in the research repository before each run): “Scoring of the submitted app (LabRat, formerly BoarLite)” LabRat (formerly BoarLite) delivery (final bundled build, Hugging Face upload, install-path check); “Claim text on the bounty” the claim text (v4.2.4 prepared, posting not recorded, to be revised for LabRat and posted with a link to the article); “Ruler v0.4 verdict run” landed (published as history; b4ef94f3); “Shaped factbook and pack publication” the shaped factbook (kept unpublished by the author); “DEVICE-FINAL: final on-phone matrix” DEVICE-FINAL, the on-phone matrix (landed 2026-10-25; the headline). Beyond closure: reading in small models, a pre-registered (not post-hoc) cross-family judge, and an item set written entirely outside both judge families.

Badge rows the 3 public fact rows this depth cites; a number here without a chip comes from a row that is not public yet (see /data/)
Claim and valueSetStatusJudgenSeeds95 % CIDate
Bar test power at n = 110: required point estimate ≈ 0.53 without run noise, ≈ 0.545 with one-seed run noise; a true 0.53 passes only ≈ 29-50 % of the time.0.53 / 0.545v0.3 plannot recordednot recorded110not recorded2026-10-12
v0.3 target composition: 110 items in the same 10 strata, ≥ 40 % centred outside Europe/North America, ≤ 2 per country, each hinging on a checkable fact.110v0.3 (unwritten)not recordednot recordednot recordednot recorded2026-10-07
Bench v0.4 exposurefirst answers by two engines 10-24; any later engine change = developed after exposurev0.4 (held-out ruler, now exposed)held-out ruler (v0.4; exposed to two engines 2026-10-24)Claude Opusnot recorded22026-10-24

In one sentence

The research is closed; what remains is the phone test of the simple app, hosting it, and posting the claim.

Turning point

2026-10-15 10:07: the record went public; 2026-10-24: the submission became the author's own app, so the remaining fronts are delivery and scoring, not research.

On the calendar

18 dated events · 3 Oct → 20 Oct

  1. 7 OctBench v0.3 spec accepted: claim set to be written outside the Claude family by the author with a non-Claude web-UI model
  2. 14 Oct · 21:59Team's closing-sprint deadline set to 2026-10-20 (the bounty page says final decision by 2026-10-31); closing sprint
  3. 14 Oct · 22:21LONGSESSION: no quality drop across 5 turns (and to turn 15)
  4. 15 OctSubmission vehicle = maintainer's native Kotlin app; PR #85 (cut 7 engine as reference) open
  5. 15 Oct · 10:10Engine PR opened to the upstream app; CI green 10:49
All 18 events
  1. 2026-10-03No more work inside the app maintainer's app; build "BOAR Bench" measurement app
  2. 2026-10-07Bench v0.3 spec accepted: claim set to be written outside the Claude family by the author with a non-Claude web-UI model
  3. 2026-10-10 17:34Claim readiness audit: evidence complete except v0.3; submission side not started
  4. 2026-10-12v0.3 power statement: bar test at n = 110 needs a point estimate ≈ 0.53 (≈ 0.545 with one-seed run noise)
  5. 2026-10-14 21:59Team's closing-sprint deadline set to 2026-10-20 (the bounty page says final decision by 2026-10-31); closing sprint
  6. 2026-10-14 22:21LONGSESSION: no quality drop across 5 turns (and to turn 15)
  7. 2026-10-15Submission vehicle = maintainer's native Kotlin app; PR #85 (cut 7 engine as reference) open
  8. 2026-10-15Port spec of the health path written for the native app
  9. 2026-10-15 10:10Engine PR opened to the upstream app; CI green 10:49
  10. 2026-10-15 11:00Official submission vehicle = maintainer's native Kotlin app
  11. 2026-10-15 22:14Ternary Bonsai models closed; Qwen3-1.7B becomes the lite candidate
  12. 2026-10-15 22:2xAuthor: lite tier is an open, post-10-20 exploration line; starts with an unsafe-answer warning
  13. 2026-10-16v0.3 write kit handed to the author (items still unwritten)
  14. 2026-10-16 09:38NUMGUARD r3 + HEALTHGAP r2 added to the upstream PR; WARN banner spec v2
  15. 2026-10-17 04:05Submitted app (maintainer's native build) scored for the first time; maintainer-supplied traces
  16. 2026-10-17 04:05Submitted native app scored internally on 57 public items (numbers held internal)
  17. 2026-10-17 23:07HONESTPROMPT launched (maintainer's honesty-prompt edits vs invented facts)
  18. 2026-10-20Bounty deadline (moved)

Related nodes on the map

This chapter on the map →

Decisions

.md