Closed record · 1.0
Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar
Status · as of 2026-10-25 16:46
What’s still open
The research lines closed on 2026-10-23. A few fronts can still move the standing; each says what would change, in both directions.
Headline · measured on the phone · as of 2026-10-25
0 of 71 wins, below the 0.50 bar
LabRat on a Pixel 8a in airplane mode, bench v0.4, two seeds, both judge families. The stricter family sets the verdict.
U-total on the phone · 0 = refusal, 1 = matches the reference
- Same as the desktop run+0.003 / −0.007paired, per judge family
- Median answer13.9 sfirst word 9.8 s
- Speed · memory11.8 tok/s3.87 GB peak, no network
From the recordAs of 2026-10-25, on a Pixel 8a in airplane mode, LabRat won 0 of 71 head-to-heads under the Claude Opus judge and 0 of 71 under OpenAI gpt-5.6-sol (via the Codex CLI). Ruler U-total: Claude 0.165 (0.127–0.205; seeds 42, 43), gpt-5.6-sol 0.242 (0.194–0.296; seeds 42, 43); the lower family, 0.165 (0.127–0.205), is below the 0.50 bar. Speed and memory: median answer 13.9 s, first word 9.8 s, 11.8 tokens/s, peak memory 3.87 GB, no network traffic.
DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.900 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 barSetbench v0.4 (exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references)n71 headline items + 4 anchorsSeeds42, 43Date2026-10-25 13:2xCaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen. One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileDEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdictpaired +0.003 (Claude) / −0.007 (gpt-5.6-sol)Setbench v0.4 (exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references)n71 headline itemsSeeds4295 % CIClaude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023Date2026-10-25 13:2xCaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen. One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileDEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permissionp50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network trafficSetbench v0.4 turns on the phoneStatusdevJudgenone (device metrics)n82 model-written turns per seedSeeds42 (43 for the warm read)Date2026-10-25 13:2xCaveatOne phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.Sourcenot in the public repositories yetOpen the data fileDEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine changematrix M1-M5; claim numbers = M1 + M5Setbench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60StatusdevJudgeClaude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5)n75 items × 3 seeds + 265 product itemsSeeds42, 43, 44 (M1); 42 (M2-M4)Date2026-10-25 03:3xCaveatNo DEVICE-FINAL result is in this record; the matrix was running as of this update.Sourcenot in the public repositories yetOpen the data file
Fronts
Pending
Claim text on the bounty
Which wording is posted to the bounty at closure, and in which vehicle?
The claim text is the public statement the bounty reads; it must match the standing on this page at the time it is posted.
What would change, quoted from the status record
The claim text follows the standing at closure: if OF-1 to OF-3 move the standing, the claim sentence moves with it; if they do not, the current sentence stands. The site quotes the posted version with its date.
Claim post v4.2.4 was prepared paste-ready on 2026-10-18; whether and when it was posted is not in the record (UNKNOWN). The submission is now LabRat (formerly BoarLite). The claim-text plan (2026-10-24 PM) leads with what the app does and the final ruler numbers labelled with both judge families, seeds, desktop twin or phone, and win-rate; wording goes to an independent reviewer, then the author. It cannot quote ruler numbers while they are internal (AD-15), and the on-phone numbers (OF-8) are not in yet. Author decision 2026-10-25: the revised claim is posted with a link to the article; corrections in the public repositories are the author's push. Update 2026-10-25 16:29: a submission post (one post with a 60 s video, numbers from DEVICE-FINAL) is prepared; as of 16:46 it is not recorded as posted. The article is live (16:46).
Pending
Shaped factbook and pack publication
Is the shaped factbook pack (v2) published, and does the pack-side lever hold beyond its 20 dev items?
Pack shaping is the one content-side lever that survived the closing tests; BoarLite ships factbook v1, not the shaped v2 (decision 2026-10-24 18:03).
What would change, quoted from the status record
If confirmed on more items, the delivery chapter gains a measured pack-side lever; if not, it stays NOT-CONFIRMED by its frozen rule (one new warned confident error) and is described as such.
NOT-CONFIRMED by its frozen rule (2 seeds, n 20, dev); dropped from BoarLite on 2026-10-24 18:03 (factbook v1 stays); the v2 card is drafted as user-selected only; publishing waits for the author. Author decision 2026-10-25: factbook v2 is not published; its verdict stays NOT-CONFIRMED and LabRat ships factbook v1 (D-196).
Landed
Scoring of the submitted app (LabRat, formerly BoarLite)
How does the submitted app score on the frozen ruler, measured on a phone, and may its numbers be published?
The submission changed twice: the app maintainer's native app (2026-10-15, reaffirmed 2026-10-21), then BoarLite, the author's own closed app (2026-10-24). Research-engine numbers do not transfer to either app.
What would change, quoted from the status record
The submitted app's numbers are public by the author's decision of 2026-10-25 (AD-15 (b)): the on-phone DEVICE-FINAL run of LabRat 7788924a (OF-8) becomes the headline in whichever direction it falls; the desktop-twin run of b4ef94f3 (one seed) stays as history with the persona-line caveat. Release-candidate scores of the app maintainer's app stay internal.
As of 2026-10-24 20:34: BoarLite final candidate b4ef94f3 built (health gate, fixed line, factbook v1, QuickMath, search port ON_NO_HEALTH) and an earlier candidate smoke-tested offline on a Pixel 8a (model ready 18 s, first answer 11.9 s). Pending: the release APK rebuilt from 6aa7508a with two UI-only commits (0 engine files) and its phone smoke, APK hosting (above a 2 GB asset limit), a ruler run on the phone. Its ruler scores exist on the desktop twin and are internal-pending-author (PD-09 / AD-15); scores of the app maintainer's RC1 and of the native engine branch stay internal. Update 20:47: the final release APK (UI-only commits on top of b4ef94f3) passed its offline smoke on the Pixel 8a; APK hosting remains. Update 2026-10-25: renamed LabRat at the app maintainer's request (7788924a, assistant Rat; same package id); the Rat persona line entered the grounding prompt after the ruler run on b4ef94f3, so that number is for b4ef94f3, not for LabRat 7788924a. Cleanup decisions: long-chat summary, PIN, Portuguese and the Expo/React Native tree removed, Settings simplified, licences in About. Hosting decided: Hugging Face (GitHub release assets cap 2 GiB; APK 3.04 GB), upload pending. Install-path risk: Android developer verification may add an 'advanced flow' for browser installs (scope unconfirmed; a test on a certified phone is planned). Open engine bug: the citation-support step is not applied to the streamed answer; a fix would need a ruler rerun. Landed 2026-10-25: LabRat scored on the phone (OF-8); LabRat 1.0.0 (dcc515fc, same engine module as the measured build) hosted on Hugging Face with its sha256 (15:3x); source public as labrat-boar-fork (15:4x).
Landed
Ruler v0.4 verdict run
What is the submitted app's U-total on the frozen 71 headline items under both judge families (Claude and OpenAI gpt-5.6-sol), measured on a phone, against the pre-registered verdict rule?
The ruler replaces the old judge.v1 ratio as the headline metric for the submitted app; until it runs, the record has no verdict number for the bounty question.
What would change, quoted from the status record
If the lower-family U-total has CI low > 0.50 with all gates passed, the standing becomes "meets the bar" for that app; if the point estimate is above 0.50 with CI low below, "above, not established"; otherwise "below the bar". In every case it is printed beside the old-scale numbers, never pooled with them.
Landed 2026-10-24 20:34 as a desktop-twin run: references of both families 75/75 (OpenAI gpt-5.6-sol half completed 20:05); BoarLite b4ef94f3 scored on the 71 headline items under both judge families, one seed, stage 1 not run, no phone run. The verdict numbers are internal-pending-author (internal_fact_ids) until the author rules on AD-15; the ruler is now exposed to three engines. Update 2026-10-25: this run measured b4ef94f3; LabRat 7788924a adds the Rat persona line to the grounding prompt after it, so its number must not be attributed to LabRat without that caveat; the on-phone re-measure of LabRat is OF-8. Update 2026-10-25 (author's publication decisions): the numbers are public as history: lower-family U-total 0.164, win-rate 0 of 71 under both families, below the 0.50 bar; the paired delta vs the native engine branch ([internal row]) stays internal because it prints that branch's absolute.
Landed
DEVICE-FINAL: final on-phone matrix
What do LabRat 7788924a's claim numbers read when the app answers on the phone itself (ruler v0.4 under both judge families with seeds stated; speed, memory and offline behaviour), under the criterion frozen before the run?
Every ruler reading so far is a desktop twin of b4ef94f3; the bounty asks about a phone, and the shipped app differs from the ruler engine by the Rat persona line.
What would change, quoted from the status record
When it lands, its claim numbers (M1 under both judge families, seeds and win-rate stated; M5 speed, memory and offline) become the publication's final standing for the submitted app, in whichever direction they fall, beside the desktop-twin reading and never pooled with it; M2-M4 enter as product numbers. Its numbers are public by the author's decision of 2026-10-25 (AD-15 (b)); M1 fills the status headline template. If the matrix does not complete, the standing stays the desktop twin of b4ef94f3 with the persona-line caveat, and the record says the on-phone run did not finish.
Landed 2026-10-25 13:2x: on a Pixel 8a, 0 of 71 wins under both judge families; U-total Claude 0.165 (0.127–0.205), gpt-5.6-sol 0.242 (0.194–0.296), seeds 42 and 43 (44 dropped for time); below the 0.50 bar; the phone repeats the desktop twin (paired +0.003 / −0.007). Interrupted once at 11:31 and resumed.
Dropped
Deep tier on a 12 GB-class phone
Does the recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) run on a 12 GB-class Android phone, at what latency and memory, and with the same judged quality as on the desktop?
Every deep-tier number in the record is desktop only; the phone time is a projection from third-party public rates, and the bounty is about a phone.
What would change, quoted from the status record
If it runs with judged parity: the deep-tier rows in 'The deep tier' and 'A phone on a desk, and in a hand' gain a measured phone scope, the projection is replaced by a measurement, and 'Where it stands (live)' can describe the deep tier as a phone configuration. If it does not fit in memory, is too slow to use, or loses quality on the phone: the deep tier stays a desktop-only result, the phone standing remains the frozen 4B build, and the attempt is recorded as a negative result.
Dropped by author decision on 2026-10-21: no 35B deep mode is delivered for the bounty; the deep tier appears only as a roadmap with its desktop numbers. No deep-tier phone run exists as of 2026-10-24; stays a projection.
Dropped
Expert-streaming engine on Android
Can the expert-streaming engine, judged equivalent to the desktop server on the 35B model, be built for Android and run within a phone's memory at a usable speed?
On a desktop CPU it runs the 35B model in under 3 GiB of RAM; on a phone only an analytic speed projection exists, and the Android build needs memory and thinking-budget work that is not done.
What would change, quoted from the status record
If it builds and runs: OF-1 gains a low-memory path, and the analytic projection is replaced by a measured phone speed in 'A phone on a desk, and in a hand'. If it does not: the phone path for the deep tier rests on the other on-device engine, and the expert-streaming result stays a desktop-only finding.
The expert-streaming engine was cross-built for Android arm64 on 2026-10-19; phone speed and memory were never measured. Dropped with OF-1 when the deep tier left the delivery (2026-10-21).
Dropped
Held-out set written outside the Claude family (bench v0.3)
What do the frozen build and the deep tier score on questions and references written outside the Claude model family and never exposed to the builders?
It is the only path to an unexposed held-out number: v0.2 is burnt and V-para is dev-style for everything after 2026-10-06. It also tests the judge-family self-preference threat at the item level, not only at the judge level.
What would change, quoted from the status record
If items are written and scored before closure: their reading becomes the held-out number in 'Where it stands (live)' and in the claim text, in whichever direction it falls, under the analysis plan's lower-judge rule. If not: the record closes stating that no unexposed held-out number exists, and the current standing stays the only held-out-at-run-time reading.
No v0.3 was written (decision D5, 2026-10-18). Its role passed to the ruler set v0.4 (open front OF-6), which is not written outside the Claude family (half Claude, half OpenAI gpt-5.6-sol). The record therefore has no unexposed held-out number written outside the Claude family; remains the only held-out-at-run-time reading on the old scale.
Changelog
Quoted from the status record. Machine-readable: /data/status.json.
- 2026-10-25 · 1.0 Final: DEVICE-FINAL landed (OF-8): on a Pixel 8a LabRat wins 0 of 71 under both judge families, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), two seeds, below the 0.50 bar; phone = desktop twin; M2-M5 product numbers added. LabRat 1.0.0 hosted on Hugging Face and its source public (OF-4 landed); pack counts stated once (13 shipped, 9 self-tested, 7 tested on the phone); article live; headline filled.
- 2026-10-25 · 0.4-preprint Author's publication decisions: LabRat (formerly BoarLite) numbers public, the app maintainer's app numbers stay internal; headline = the final ruler on the phone (DEVICE-FINAL, templated, running); the desktop ruler run of b4ef94f3 published as history (lower-family U-total 0.164, 0 of 71 wins, below the 0.50 bar); thesis restated; Citations.applyShown documented as a known limitation.
- 2026-10-25 · 0.4-preprint BoarLite renamed LabRat (assistant Rat) at the app maintainer's request; the Rat persona line entered the grounding prompt after the ruler run (ruler number = b4ef94f3); cleanup decisions (long-chat summary, PIN, Portuguese and the Expo/React Native tree removed; Settings simplified; licences in About); logo n8; APK hosting decided (Hugging Face); install-path risk noted; DEVICE-FINAL criterion frozen and the on-phone matrix started (OF-8, running; interrupted once by an APK install and restarted); open engine bug Citations.applyShown recorded.
- 2026-10-24 · 0.3-preprint 20:47: final BoarLite release APK (6aa7508a, engine b4ef94f3) passes its offline smoke on a Pixel 8a.
- 2026-10-24 · 0.3-preprint Evening (17:39-20:34): BoarLite final candidate b4ef94f3 (health gate B+FU, fixed line, factbook v1, QuickMath, preload, onboarding); device smoke on a Pixel 8a (unpack 14 s, 0 B network, PSS 3.09 GB, first answer 11.9 s); SEARCHPORT failed its frozen safety gate and its grounded-dose re-gate, then merged as ON_NO_HEALTH by the author's prior condition although Addendum 2 failed S1 by the letter; GROUNDED-DOSE rule (arms B, C pass); Wikipedia-health limitation accepted; second-family references 75/75; final ruler run on BoarLite's engine landed (OF-6), numbers internal pending the author (AD-15).
- 2026-10-24 · 0.2-preprint Validation judge keeps Succinct; author decides a closed, simple, Succinct-only chat app and declares BoarLite (his own app) the submission; first ruler v0.4 answers (desktop twins, Claude half only, internal); status reframed: research closed, delivery and ruler run open.
- 2026-10-23 · 0.1-preprint Author closes the research lines (11:53); closing tests (UXMATRIX, PACKPROOF, U5, WARN-KEPT, NOWIKI, PACK-GAP, QREWRITE); thesis held only in a conditional form; CONTENT-FIRST pack work (shaped factbook); ENGINE-RC built and validated overnight.
- 2026-10-22 · 0.1-preprint Direction: packs and delivery are the strategy; small focused packs; delivery counted at chunk level; search fixes SEFIX-124 adopted for delivery (+0.013, not a lever; F3 withdrawn as unsafe).
- 2026-10-21 · 0.1-preprint RULER-SPEC v1 r3 and r4 frozen (judge.v1 + DECLINE2STAGE, A∪ verdict arm, two judge families, information-available split); author: no 35B deep mode delivered, submission = the app maintainer's native app; release-candidate scores kept internal.
- 2026-10-20 · 0.1-preprint Pivot: Qwen3-1.7B becomes the main 8 GB engine (4B optional tier) on phone latency and memory; bench v0.4 item set frozen (75 items) before any system answered; refuse-by-default product rule.
- 2026-10-19 · 0.1-preprint V-para diagnosis: the 4B gap is mostly wrong-from-memory and absent knowledge; independent direction review says stop opening arms. Public ruler designed (BENCH-RULER).
- 2026-10-18 · 0.1-preprint Author delegates seven open decisions (D1-D7); adopted: no bench v0.3 before closure, claim v4.2.4 prepared, site uploaded only as v1.0 at closure. Honest-prompt line closed (HONESTPROMPT).
- 2026-10-18 · 0.1-preprint First living-preprint version: content reframed as interim; status banner, standing and open fronts added.