> Closed record · research record 1.0 · as of 2026-10-25 16:46; the author closed the research lines on 2026-10-23. Current standing, not a final result: Open fronts: https://r4topunk.github.io/boar-research/data/status.json.
>
> Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

Markdown twin of the article at /. Chapters: https://r4topunk.github.io/boar-research/research/ · Whole paper: https://r4topunk.github.io/boar-research/llms-full.txt · Index: https://r4topunk.github.io/boar-research/llms.txt · Data: https://r4topunk.github.io/boar-research/data/.

*RatLab — A Boar Research · r4to, independent researcher · about 10 minutes*

# Can a phone with no internet answer like the web plus a frontier AI?

Put your phone in airplane mode. Ask it why your bread didn't rise, or which languages are official in Burkina Faso. Nothing leaves the phone: no search engine, no cloud, no big AI on the other end.

A public bounty asked for exactly that phone, and we spent weeks finding out what it takes. The short version: **a small model is only as good as the page you hand it.** On questions written from its own library, when search put the right article in front of our small model, it was right about two times in three pack-written · 1 seed [F-1705]. So we stopped trying to make the model smarter. We built everything around getting it that page, and around keeping it honest when the page doesn't come.

This is the story of how we learned that, the ruler we had to build first, the many ideas that died, and the app that came out of it. [sentence withheld in this file; see https://r4topunk.github.io/boar-research/data/ or the chapter record]

> Chart (hero): everything the phone knows is on the shelf inside it, and the real work is fetching the right page. [F-2224] [F-1386] [F-2235]

> **For judges**
>
> 1. **What it is:** LabRat, an offline Android research assistant with the Qwen3-1.7B model, openly licensed knowledge packs and no internet permission, in one APK app record [F-2224] engine choice [F-1386] APK record [F-2235].
> 2. **Result:** on a Pixel 8a in airplane mode, LabRat won 0 of 71 questions against the reference under both judge families. Its U-total is 0.165 (0.127–0.205) with Claude Opus and 0.242 (0.194–0.296) with OpenAI gpt-5.6-sol, on a 0–1 scale where 0.50 is the bar: below the bar phone · 2 seeds [F-2238]. The phone matches the desktop run of its pre-rename engine paired · 1 seed [F-2239]. Criterion frozen before any answer frozen criterion [F-2227].
> 3. **Claim:** small models need external knowledge, and the app is built around what a small model can really do author's thesis [F-2236]. As of 2026-10-25 it is not shown to be half as good as frontier AI with web.
>
> **Install:** [huggingface.co/r4topunk/labrat](https://huggingface.co/r4topunk/labrat) · sha256 `d2e00ba4a9adff6f903eab7d34759fd4f4db9cd721303dc7c38ae280b465dae8` release record [F-2246] · 3.04 GB, about 6 GB free, Android 8+, allow "install unknown apps", accept the Play Protect prompt install notes [F-2230]. Android's new developer checks may add a warning on some phones; untested on a certified phone as of 2026-10-25 open risk [F-2231].
>
> **Verify:** the old judge, its scorers and the frozen analysis plan are in the public boar-bench repository; the second ruler's spec and scorer are planned to join them. Tap any number below for its source row in [/data/facts.json](https://r4topunk.github.io/boar-research/data/facts.json).
>
> **Everything public:**
>
> - [github.com/r4topunk/labrat-boar-fork](https://github.com/r4topunk/labrat-boar-fork): the LabRat app source, the submission repo record [F-2247]
> - [huggingface.co/r4topunk/labrat](https://huggingface.co/r4topunk/labrat): the APK
> - [github.com/r4topunk/boar-research](https://github.com/r4topunk/boar-research): the research record (methods, frozen criteria, negative results, device evidence)
> - [github.com/r4topunk/boar-bench](https://github.com/r4topunk/boar-bench): the benchmark, the judge harness and the trace format
> - [huggingface.co/datasets/r4topunk/boar-packs](https://huggingface.co/datasets/r4topunk/boar-packs): the knowledge packs

## 1. The question

The bounty grew out of a public post by Vitalik Buterin asking for an offline research tool on a phone public post [F-047]. Poidh bounty #31 was opened later, inspired by that post. He did not issue it and is not affiliated with it, though his public confirmation that a build meets the target would win it bounty text [F-116].

The rules are concrete. An Android phone with at most 12 GB of RAM, no connection and no Google Play Services bounty text [F-071]. Answers "more than 50 % as good as internet plus a frontier AI". The bounty page gives a public decision date of 2026-10-31 bounty page [F-040]. We chose to work in English only team scope [F-075].

That target hides a hard question. Half of what? Measured how, by whom, on which questions? If you grade a student's essay against a professor's, "half as good" depends on who grades and what counts. So before improving anything, we had to build a ruler we could trust.

## 2. The ruler

We wrote questions and, for each one, froze a reference answer from a frontier model with live web search. The references were hashed and never regenerated, so every later run faces the same opponent method [F-059].

An AI judge then reads our answer and the reference side by side. Links and citation marks are stripped first, so it cannot simply spot the answer that came from the web method [F-649]. It scores both from 1 to 5 on correctness, completeness and usefulness rubric [F-125]. Then we swap the order and ask again. If its pick flips, the question counts as a tie rule [F-235].

> Chart (judges-desk): the judge cannot see which answer is ours, has to agree with itself in both orders, and is checked by a second judge from another family. [F-649] [F-125] [F-235] [F-017]

Two choices matter most.

**Two judges from different families.** Our main judge is Claude, and the reference answers were written by Claude too. A judge might favour writing from its own family, and a second Claude judge cannot catch that method check [F-1216]. So we added a judge from OpenAI, gpt-5.6-sol through the Codex CLI. Run after the fact, it scored both systems it re-judged 0.08 to 0.14 higher, but kept the same ranking and almost no wins dev-style · 2 seeds · run after the fact [F-017]. A rule fixed in advance says the stricter judge sets the headline rule [F-1239].

**Rules before results.** Before any run that decides something, the pass/fail rule is written down and its fingerprint (a sha256 hash) recorded. An audit matched 65 of 65; an outside critic later found 16 of 17 matching audit [F-1169]. Every look at a held-out set, the questions kept unseen while building, was logged. That log is also how we know both early held-out sets later became practice data exposure log [F-038].

For the app's verdict we built a second ruler, bench v0.4: 75 questions frozen before any system answered them frozen 2026-10-20 [F-2070], written roughly half by each judge family writers [F-1416], with references from both families Claude refs [F-2110] OpenAI refs [F-2212]. On it, a refusal scores zero design [F-1836]. Our engines have answered it since 2026-10-24, so it is no longer unseen exposed [F-2214].

## 3. The floor trap

The old score was a ratio: our answer's points divided by the reference's points. It looks fair. It is not, because the rubric starts at 1, not 0.

An app that politely declines every question still scores about 0.31 dev · 7 runs [F-003]. The references average about 4.9 out of 5 dev-style [F-182]. So a blank answer gets more than halfway to the 0.50 mark for free.

> Chart (floor-trap): on the raw scale silence already scores, so we grade on the scale where silence is zero. [F-003] [F-236] [F-001]

The fix is simple. Subtract the free point before dividing. On this "floor-adjusted" scale, silence scores 0 and matching the reference scores 1. We also count wins: how often the judge preferred our answer over the reference, in both orders. Wins are hard to flatter. The second ruler fixed the trap at the source by scoring refusals as zero.

Here was our starting point. The frozen build of the earlier engine, on 51 bounty-style questions it had never seen, scored 0.261 floor-adjusted (0.413 raw) and won 0 of 51 held-out then · 2 seeds [F-001]. About a quarter of the way from a blank answer to the reference.

## 4. The cemetery of ideas

Then we tried to move that number. Hundreds of experiments went into an open registry ([/data/experiments.json](https://r4topunk.github.io/boar-research/data/experiments.json)). More than half closed without a gain. Many died cheaply: of the first 101 ideas we closed, 54 died before an AI judge ever saw them, killed by a cheaper check registry [F-915].

> Chart (closed-ideas): most ideas failed, the failures were cheap and recorded, and they pointed to the one that mattered. [F-915]

The walls we hit:

- **Other small models.** Swapping in other 4–9B models did not help dev · 1 seed [F-450].
- **Better retrieval.** A new retriever put the right sentence in front of the model more often, 7 → 18 of 56. The answers were not judged better dev · 2 seeds [F-443].
- **Longer answers.** Asking for them doubled confident wrong answers, 10 → 20 dev · 1 seed [F-433].
- **Fine-tuning.** Even human-written training data had unsupported claims in 6 of 20 examples. We never trained reviewer check [F-161].

The biggest lever we measured was a much larger model, 35B, that thinks before it answers. On a desktop it reached 0.436 floor-adjusted, still with 0 wins in 51 dev-style · 2 seeds · desktop [F-009]. It was the best of at least eight configurations tried on that same set, so the number is likely 0.01–0.02 optimistic. It was never run on a phone; projected from third-party rates, it would need a 12 GB-class phone and 2–3 minutes per answer projection [F-035]. The author decided not to ship it author decision [F-1532].

The app's engine became Qwen3-1.7B. It was chosen for speed and memory on an 8 GB phone, not for quality author decision [F-1386]. The larger 4B scores clearly better dev · 1 seed [F-2077], and we say so.

## 5. What worked: the harness

The turn came when we tested the packs directly. Packs are openly licensed libraries stored on the phone: first aid, cooking, an encyclopedia, Q&A sites. LabRat ships 13 of them, all downloadable from the boar-packs dataset; the self-tests below cover nine packs, seven of which are in the app, and the phone run tested those seven pack counts [F-2248]. On questions written from each pack, the small model was right 48 % of the time with the matching pack installed and 26 % without pack-written · 1 seed [F-1704]. On the same questions, when search delivered the question's own article, it was right 65 % of the time; when it did not, 28 % pack-written · 1 seed [F-1705].

But search delivered that article only 54 % of the time pack-written · no judge [F-1706]. And when we traced the facts the app failed to bring to the model, 44 of 50 were already on the phone dev · no judge [F-2100]. The library usually had the book. Opening it at the right page was the problem.

> Chart (harness): the model is one station of seven, and most of the work is getting the right page through the doorway. [F-2236] [F-1705] [F-1706] [F-2210] [F-2200] [F-2211]

That became the project's thesis: small models need external knowledge author's thesis [F-2236]. We learned how much, and built a harness: code around the model, tuned to what it can really do and to the person using the app.

- **Better delivery.** We ported the research bench's search fixes into the app, switched off on turns it reads as health. Delivered key facts went from 26 to 40 of 92 dev · 1 seed · no judge [F-2207]. The port failed its frozen safety rule (6 against 4 unsupported dose figures on the 10 health probes whose sources changed) and was merged under a condition the author had set before the run: accept the health loss if it was worth it for the rest author decision [F-2209]. No probe changed its final health path.
- **Short answers.** Longer styles were not right more often: 42 % against 43 % for the short one. They showed a wrong answer with no warning about twice as often: 23–25 per 100 against 12 dev · 1 seed [F-2106] dev · 1 seed [F-1728]. The app has one short style.
- **A line that admits it.** Every model answer ends with "May contain errors; check the source." app record [F-2211]
- **A health gate.** When no health source is found, the app shows a fixed safe text with the emergency number instead of a model answer. On 65 written safety probes, the same ones its rules were tuned on, it did so 41 times, including all 4 where the model had invented a dose; 23 still got a model answer dev · no judge [F-2210] dev · no judge [F-2192]. AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- **Sums in code.** Arithmetic and unit conversions are computed without the model, never on health turns app record [F-2200].
- **Honest gaps.** With no relevant source, the model is told to say so and answer only briefly.

The limits, once each. Those 48 % and 65 % come from questions written from the packs, which favours the packs. On 10 real practice questions the pack effect did not reproduce: −0.010, against +0.139 on 27 pack-written ones dev · 1 seed · after the fact [F-2080]. None of nine packs reached the 75 % mark we fixed in advance; the best reached 60 % pack-written · 1 seed [F-1707]. And health questions whose only source is an encyclopedia article also get the safe text, 12 of 60 health tests known limitation [F-2206]. The hardest limit is in the next section: on the new ruler, the app's engine is far below the bar.

## 6. The honest number, as of 2026-10-25

The headline is the second ruler, bench v0.4, measured on the phone itself: LabRat on a Pixel 8a in airplane mode, two seeds under both judge families (a third seed was dropped for time; two is the ruler's spec). Its rule was frozen before a single answer was generated frozen criterion [F-2227]. The run was interrupted once by an app install on the phone and resumed process note [F-2244].

**The result.** LabRat won 0 of 71 questions against the reference under the Claude judge and 0 of 71 under gpt-5.6-sol. Its U-total (desktop twin of engine b4ef94f3) is 0.165 (0.127–0.205) with Claude Opus and 0.242 (0.194–0.296) with OpenAI gpt-5.6-sol. The lower family is 0.165: below the 0.50 bar phone · 2 seeds [F-2238]. The phone repeats the desktop run of its engine: +0.003 under Claude and −0.007 under gpt-5.6-sol, paired on the same questions paired · 1 seed [F-2239].

On the same phone: right on 40 % of 140 questions written from its own packs pack-written · 1 seed [F-2242]; 53 of 65 health and safety probes got the fixed safe text, and 2 model answers added a dose figure nobody asked for phone · no judge [F-2241]. AI-reviewed, not clinically reviewed. Median answer 13.9 s, first word in about 10 s, about 12 tokens per second, 3.9 GB of memory at peak, no network traffic phone · 1 seed [F-2240].

> Chart (standing): three different kinds of number, which must not be compared on one scale, and the one that counts is the phone's. [F-2238] [F-001] [F-009] [F-1704] [F-1707] [F-2215]

**History, kept apart.** As of 2026-10-25 the 0.50 bar has not been reached under the Claude judge on any held-out set. Every reading above 0.50 is on practice data: from the unshipped desktop model, with an uncertainty range straddling 0.50, or from the second judge, run after the fact dev · 1 seed · desktop [F-013] dev · 2 seeds · run after the fact [F-488].

Before the rename, the second ruler was run once on a desktop twin of the app's engine b4ef94f3: the same answer path, run on a computer instead of the phone desktop twin [F-2214]. Its score, U-total, is the share of the reference's useful content an answer delivers, where a refusal scores 0 and a full match 1. That desktop twin of b4ef94f3 won 0 of 71 head-to-heads under both judges, and its U-total under the stricter judge was 0.164, below the 0.50 bar desktop twin · 1 seed [F-2215]. Where the packs held the answer, the same b4ef94f3 desktop twin scored about what it scored where they did not, a U-total of 0.164 against 0.155 desktop twin · 1 seed [F-2217]: with the answer on the phone, the app often did not deliver it. That run came before the Rat persona line entered the prompt, and the engine code changed afterwards code note [F-2225] code note [F-2223]. It is history, not the app you install.

## 7. The app: LabRat

The submitted app is LabRat (formerly BoarLite), and its assistant is called Rat app record [F-2224]. On 2026-10-24 the author made his own app the submission in place of the app maintainer's author decision [F-1703].

> Chart (labrat-screens): one plain chat that answers from what is on the phone, shows where each answer came from, and says when it should not guess. [F-2224] [F-2198] [F-2201] [F-2235]

It is one simple chat with one short answer style. As of 2026-10-25, everything it knows ships inside a single APK of about 3 GB, and it has no internet permission APK record [F-2235] install notes [F-2230]. On a Pixel 8a with 8 GB of RAM in airplane mode, an earlier build unpacked in 14 s, sent and received 0 bytes, and used 3.09 GB of memory after five answers phone · hand check [F-2198]. A later candidate was ready 18 s after launch and gave its first answer in 11.9 s, at 14.9 tokens per second phone · hand check [F-2201]. These are hand checks on one phone; the measured numbers are in section 6 phone · 1 seed [F-2240].

LabRat 1.0.0 is hosted on Hugging Face, because it is larger than GitHub's release limit, and its source is public release record [F-2246] repo record [F-2247]. It has the same engine as the build measured on the phone.

One known limitation. The step that checks citations against sources is not applied to the streamed text the chat shows. The author chose not to fix it before the phone run, so the engine we measure is the one you install open bug [F-2232] author decision [F-2237].

## 8. How it was made

One human and a lab of AI agents. Research agents work in fixed lanes, write one card per idea and never run experiments; only a coordinator queues runs lab rule [F-823]. Rules are hashed before their runs, every look at a held-out set is logged, and wording decisions go to an independent reviewer agent, not to whoever ran the test. A four-day unattended research loop produced exactly one practice-only lever loop log [F-031]. We report that too. The registry of dead ideas may be the most reusable thing we made.

## 9. Go deeper

- **Repositories:** [app source](https://github.com/r4topunk/labrat-boar-fork) · [APK](https://huggingface.co/r4topunk/labrat) · [research record](https://github.com/r4topunk/boar-research) · [benchmark and judge](https://github.com/r4topunk/boar-bench) · [knowledge packs](https://huggingface.co/datasets/r4topunk/boar-packs).
- **The full research:** 18 chapters, each at three depths (curious, technical, AI pros), with the interactive figures: [chapters](https://r4topunk.github.io/boar-research/research/).
- **The map:** the ideas, experiments and decisions, linked by "led to", "ruled out" and "uses": [map](https://r4topunk.github.io/boar-research/map/).
- **Ask your AI:** hand your assistant [/llms.txt](https://r4topunk.github.io/boar-research/llms.txt), [/llms-full.txt](https://r4topunk.github.io/boar-research/llms-full.txt) and the open data in [/data/](https://r4topunk.github.io/boar-research/data/). Every number has a source row, so it can cite instead of guess.

Text CC BY 4.0; code Apache-2.0; data under each source's licence. Health content is AI-reviewed, not clinically reviewed.

## Facts cited {#facts}

Resolve every [F-…] marker in https://r4topunk.github.io/boar-research/data/facts.json. Each line gives the claim, the number with its metric family, set and status, judge, n, seeds, CI, date and caveat.

<!-- lint:exempt:start -->
- [F-001] The frozen 4B claim build on the bounty-style set (V-para) delivers about a quarter of the frontier reference's value above "nothing", and loses every head-to-head. raw 0.413 (0.384–0.445); floor-adjusted 0.261 (0.224–0.300); win/tie/loss 0/0/51 in both seeds (judge.v1 ratio) · set V-para (held-out for this run (cut 3 on 2026-10-05, cut 6 on 2026-10-06); dev-style for everything measured after 2026-10-06) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 (42, 43) · CI 95 % item bootstrap · date 2026-10-06 · caveat: Cut 7 = cut 6 on this set (the FACTS table fires 0 times on it). Public BAR-TABLE prints CI high 0.300; one claim draft prints 0.299 (rounding).
- [F-003] A system that declines every question already scores about 0.31 on the raw ratio, because the rubric never goes below 1 of 5. always-decline raw ratio D = 0.306 (v0.2-weighted 0.307) (judge.v1 ratio) · set v0.1 night runs (dev, development data) · judge Claude Opus (judge.v1, both orders) · n 79 judged declines over 7 runs · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: This is why the floor-adjusted ratio, not the raw ratio, is the headline.
- [F-009] Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set. raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 s (judge.v1 ratio) · set V-para (dev-style, development data) · judge Claude Opus (judge.v1, both orders) · n 51 · seeds 2 · CI 95 % item bootstrap; common-mode CI adds seed noise · date 2026-10-09 · caveat: Chosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only.
- [F-013] Recommended deep configuration on v0.2: the first point above 0.50 floor-adjusted on any set, with a CI that straddles 0.50. raw 0.608 (0.575–0.641); floor-adjusted 0.508 (0.467–0.550); win/tie/loss 0/1/109; desktop p50 ≈ 82 s (judge.v1 ratio) · set v0.2 (dev (burnt), development data) · judge Claude Opus (judge.v1, both orders) · n 110 · seeds 1 (42) · CI 95 % item bootstrap · date 2026-10-09 · caveat: One seed, development data, desktop only.
- [F-017] A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate. 4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01 (judge.v1 ratio) · set V-para (dev-style, development data) · judge cross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered) · n 51 (408 judge calls) · seeds 2 · CI 95 % item bootstrap (4B floor-adjusted CI not in the sources read) · date 2026-10-16 · caveat: Post-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline.
- [F-031] Four days of an unattended agent research loop produced one dev-only lever. 133 ticks (28 idle), 2026-10-11 12:10 → 10-14 19:09; ≈ 10.5k judge calls (Opus 4,851 + Haiku 5,660) plus 1,491 Haiku writer/generation calls; Mac mini 44.4 h + MacBook 18.1 h of generation; ≈ 31 h avoidable · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14
- [F-035] Projected phone time for the recommended deep configuration (never measured on a phone). ≈ 1.9–2.8 min per answer at thinking budget 512 on a 12 GB phone CPU (third-party public decode rates of 4–6 tok/s + BOAR arithmetic); the 2026-10-09 estimate of 3.5–5 min was for budget 1024 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI projection · date 2026-10-12 · caveat: Third-party rates, not a BOAR measurement.
- [F-038] Every touch of a held-out set was logged; both held-out sets are now development data. v0.2: 12 full generation runs, 10 full Opus passes → burnt, dev (ruling 2026-10-06); V-para: 19 full runs + 12 slices → held-out for the 4B claim runs only, dev-style after 2026-10-06 (ruling 2026-10-09) · set v0.2; V-para (burnt / dev-style, development data) · judge UNKNOWN · n 110 / 58 (51 judged + 7 venue) · seeds UNKNOWN · CI not computed · date 2026-10-09
- [F-040] Deadlines: the bounty page (swept 2026-10-13) gives a final decision by 2026-10-31, and the issuer has said there is no fixed deadline; 2026-10-20 was the team's own closing-sprint deadline. 2026-10-31 (bounty page); 2026-10-20 (closing sprint) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: UNKNOWN whether the bounty itself moved to 10-20; call 10-20 the team's closing-sprint deadline unless the author confirms.
- [F-047] Public post (by Vitalik Buterin) asks for an offline phone research tool ">50 % as good as internet + frontier AI"; bounty #31 is later opened on poidh, inspired by it. · date 2026-09-17
- [F-059] References = Claude Opus + web search, frozen once, hashed; never regenerated (why: a fixed denominator makes runs comparable) · date 2026-09-30 → 10-02 · caveat: outcome: same refs used for every v0.1 number; regenerating = not comparable
- [F-071] The bounty's hardware envelope. ≤ 12 GB RAM, ≤ 50 GB total storage, offline, no Play Services, GrapheneOS-compatible · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02 · caveat: GrapheneOS not yet run on hardware
- [F-075] Single goal = win the bounty; English only; no more work inside the maintainer's app; build a dedicated measurement app — why: focus — outcome: PT/ES lines parked; BOAR Bench built. · date 2026-10-03
- [F-116] Bounty re-read: winning is a public confirmation, not a numeric benchmark. · date 2026-10-02
- [F-125] The judge scores each answer 1-5 on three axes: correctness (a material wrong claim caps it at 2; an uncorrected false premise at 1), completeness, usefulness. 3 axes, 1-5 · set all sets (status UNKNOWN) · judge judge.v1 · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-02
- [F-161] Fine-tune data under one strict faithfulness rubric: teacher 16/20, + sentence rejection 11/20, human CLAPnq+HAGRID 6/20 examples with an unsupported claim. 16 / 11 / 6 of 20 · set n/a (status UNKNOWN) · judge Opus reviewer · n UNKNOWN · seeds UNKNOWN · CI 20-example reviews · date 2026-10-07 · caveat: bar ≤ 2/20 unreachable even for human data
- [F-182] The reference answers score ≈ 4.9/5 on average, so the ratio behaves like an absolute rubric mean rather than a head-to-head. ≈ 4.9 (judge.v1 ratio) · set v0.2, V-para (status UNKNOWN) · judge Opus · n 51 (inferred from set) · seeds UNKNOWN · CI not computed · date 2026-10-10 · caveat: public citation fix: per-dimension reference means (4.63-5.00) are at l.88-91, not l.66-69
- [F-235] Combining the two orders: same winner keeps it, disagreement becomes a tie; rubric scores are averaged. rule · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed
- [F-236] Raw ratio = Σ S / Σ R, where S and R are the app's and reference's mean rubric (mean of 3 axes, both orders averaged). formula (judge.v1 ratio) · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed
- [F-433] Asking the 4B for longer answers (persona P2) lowered quality: correct items 39 % → 14 %, confident-wrong 10 → 20. Δ −0.044 · set v0.1 dev (dev, development data) · judge Opus · n UNKNOWN · seeds UNKNOWN · CI (−0.076…−0.014), MDE 0.029 · date 2026-10-02 · caveat: judge length slope negative (−0.34 per log-word)
- [F-443] Retrieval v2 put the gold sentence in context 7 → 18/56 times but lowered quality. Δ −0.021 · set v0.1 EN dev (dev, development data) · judge Opus · n UNKNOWN · seeds 2 · CI CI90 −0.054…+0.009, 2 seeds · date 2026-10-03 · caveat: retrieval 3.6× slower; 3 items carry the loss
- [F-450] Bigger closed-book models do not help the 4B tier. Qwen3.5-4B 0.437, Gemma E4B 0.495, Qwen3.5-9B 0.488, Ministral 8B 0.457 vs 0.487 · set v0.1 dev (dev, development data) · judge Opus · n 57 · seeds 1 · CI n 57; 1 seed · date 2026-10-05/06 · caveat: not detected at MDE 0.055
- [F-488] Cross-family judge on v0.2: 4B raw 0.617 (0.567-0.666) / floor-adj 0.517 vs Claude 0.512 / 0.388; deep raw 0.714 (0.666-0.759) / floor-adj 0.639 (0.579-0.695) vs 0.608 / 0.508. ≈ +0.10 raw / +0.13 floor (judge.v1 ratio) · set v0.2, burnt → dev (burnt → dev, development data) · judge cross-family vs Opus (post-hoc, different judge family) · n 110 · seeds 2 · CI 660 calls, n 110; 4B 2 seeds, deep 1 seed · date 2026-10-17
- [F-649] Blinding strips think blocks, trailing "Sources:" lists, URLs, markdown links, [n] markers and bold before judging. 1 function · set all (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: stripping sources also removes web provenance signals from the reference
- [F-823] Research agents work in fixed lanes, write one "technique card" per idea, never run experiments or touch the product; only the coordinator queues runs — why: Avoid duplicate work; separate proposing from measuring — outcome: the retrieval research agent 19 cards, the models/latency research agent 22, the evaluation/safety research agent 23; runner agent executed. · date 2026-10-01
- [F-915] How arms died. 101 closed arms: 54 at 0 judge calls, 10 at Haiku L1, 12 at L1→Opus, 25 Opus directly · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-14 · caveat: citation fix
- [F-1169] Criterion hashes verified: 65/65 on 2026-10-10; an outside critic found 16 of 17 matching on 2026-10-16. 65/65 (10-10); outside critic: 16/17 criterion hashes match (verdict), 19 of 20 in the C6 spot-check (1 mismatch by hashing convention, 1 without a .sha256 sidecar) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-10, 2026-10-16 · caveat: some early freezes attested by mtime only
- [F-1216] Changing the judge to another Claude model: Fable second judge. moves the ratio < 0.01 (judge.v1 ratio) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · caveat: citation fix (l.27 is a table header); why: does not address self-preference
- [F-1239] Claude judge stays the headline (lower of two judges); judge family named next to every ratio; win/tie/loss leads — why: analysis-plan rule: when judges differ by > 0.05, the lower is the headline — outcome: claim v4.2.x. · date 2026-10-16/17
- [F-1386] Qwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tier. adopted; confirmed at 19:07 grill · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 ~11:50 · caveat: who decided: author; why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill; alternatives: keep 4B default; Qwen2.5-1.5B (faster but weaker); reversible: yes
- [F-1416] v0.4 candidate writers. Claude 36, OpenAI gpt-5.6-sol 34; 302/307 points supported, 5 disputed · set v0.4 drafts (frozen ruler set (held-out for the ruler; counts only)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 10-20
- [F-1532] 35B deep mode not delivered; deep tier appears only as roadmap with desktop numbers. stood · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: who decided: author (GRILL); why: focus on 8 GB UX; no Android expert-streaming build; alternatives: 12 GB deep tier; reversible: yes
- [F-1703] BoarLite = a NEW bounty submission by the author (not the app maintainer's app); closed app, everything bundled, full K13 kit in one APK, shaped factbook v2, port the warning line, Qwen3-1.7B fixed. Implementation next; APK ≈ 3.0 GB, hosting open · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 · caveat: who: author; why: (decision page); reversible: partly (submission identity) · later: 2026-10-24 18:03: factbook v1 stays (D3 reverted) and a fixed line on every answer replaces the ported warning line (D4); final candidate b4ef94f3 at 19:40.
- [F-1704] PACKPROOF pooled correct-any with pack installed vs without. 0.48 vs 0.26, Δ +0.23 (outcome-label share) · set 180 pack self-tests (written from the packs), dev (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels; correct-any · n 180 · seeds 42 · CI CI +0.16..+0.31; n 180; seed 42 · date 2026-10-23 · caveat: self-tests favour the pack; hosts differ by arm; 1 seed Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.
- [F-1705] Correct-any when own article is kept vs not. 0.65 (n 98) vs 0.28 (outcome-label share) · set 180 pack self-tests (written from the packs), dev (development data) · judge Claude Haiku labels · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-23 · caveat: "about 2 in 3 when delivered" | 'kept' = own article kept (article level), between-item split, n 98 kept.
- [F-1706] Own article delivered (kept) 54 % ; cited 28 % · set 180 pack self-tests (written from the packs), dev (development data) · judge 0-LLM funnel · n 180 · seeds UNKNOWN · CI n 180 · date 2026-10-23 · caveat: delivery = bottleneck | 'kept' here is article-level (own article in the kept list), not the chunk-level delivery used for MAIN facts since 10-22 ; do not mix with the 42/92 chunk-level count.
- [F-1707] Packs reaching the 0.75 "proves" bar. 0 of 9; 7 below 0.60 (not default-eligible by rule); best 0.60 (two Q&A packs), worst factbook 0.25 · set 180 pack self-tests (written from the packs), dev (development data) · judge Claude Haiku · n 20 · seeds UNKNOWN · CI n 20 per pack · date 2026-10-23 · caveat: health packs AI-reviewed, not clinically reviewed AI-reviewed, not clinically reviewed.
- [F-1728] S+ / two-paragraph personas. correct-any ON 0.42 / 0.42; WU ON 0.23 / 0.25 (vs 0.12) (outcome-label share) · set UXMATRIX 60, dev (development data) · judge Claude Haiku · n 60 · seeds 42 · CI n 60; seed 42 · date 2026-10-24 · caveat: warning line 0.60→0.48 with longer answers Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.
- [F-1836] Adopt the robust ruler design (true-zero U, uncapped two-family refs, items outside the judge family, app on phone) became RULER-SPEC v1 · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-19 · caveat: who: delegated coordinator (8-agent workflow); why: five misreadings of the bounty in the old bench; reversible: yes
- [F-2070] Bench v0.4 frozen. 75 items (71 headline + 4 anchors); 329 checklist points supported on the open web (at least one family, none contradicting; the 32 balance-round points by both families); sha f9ad2f2f (ruler U-total) · set v0.4 (held-out ruler) (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-20 · caveat: written by Claude + OpenAI gpt-5.6-sol (the judge families)
- [F-2077] Tier ladder under Opus. 4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170) · set v0.1 (dev) (dev, development data) · judge Claude Opus judge.v1 · n UNKNOWN · seeds UNKNOWN · CI v0.1 51, 1 seed per rung · date 2026-10-21 · caveat: Haiku: 1.7B−1.5B +0.038 (−0.023..+0.098), so judge-dependent
- [F-2080] Pack self-test gain does not reproduce on real items. self-test fact-added cell +0.139 (+0.075..+0.201) n 27 vs real −0.010 (−0.077..+0.050) n 10 · set self-tests (pack-written) vs MAIN keyed (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 1 seed · date 2026-10-21 · caveat: post hoc; small n on real
- [F-2100] Undelivered facts are a search problem. 44 of 50 already in the K13 kit · set dev (dev, development data) · judge 0 LLM · n UNKNOWN · seeds UNKNOWN · CI 50 items · date 2026-10-23
- [F-2106] Longer styles do not add correctness. Succinct WU ON 0.12 vs S+ 0.23, P2 0.25; correct-any 0.43 / 0.42 / 0.42 (outcome-label share) · set dev (dev, development data) · judge Claude Haiku · n UNKNOWN · seeds UNKNOWN · CI 60 items · date 2026-10-24 · caveat: Metric: outcome-label shares of n items (Claude Haiku 5.5-low labels, UXMATRIX protocol); not the judge.v1 ratio.
- [F-2110] Claude references for v0.4 frozen before any app answer. 75/75 items, 88 turns, uncapped, median 703 words, median 6 sources · set v0.4 (held-out ruler (v0.4; exposed to two engines 2026-10-24)) · judge Claude Opus + web · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-21 · caveat: OpenAI gpt-5.6-sol half 15/75 by 10-24
- [F-2192] Health detector B+FU (prototype): the app's lexical health test OR 11 lexical rules (drug names, dose units and events, vitals, child fever temperature, symptoms, poisoning, alcohol, pregnancy, child dosing, trauma, altitude, diving) OR a short follow-up of a health turn; it gates 37 of 65 probes, including all 4 answers that had invented a dose figure, with 0 new fires on 251 non-health dev items and 1 of 40 adversarial items. 37 / 65 gated; 4 / 4 invented-dose answers; new false fires 0 / 251 and 1 / 40 · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes; 251 + 40 non-health items · seeds UNKNOWN · CI not computed · date 2026-10-24 18:23 · caveat: Rules fitted on the same 65 probes (in-sample). Rejected: title overlap with health articles (gated 5 of 19 health self-tests) and a port of the mmBERT health detector (106 MB, 2-3 days). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2198] Device smoke of BoarLite (APK 9d7fdc2e) on a Pixel 8a in airplane mode: first launch unpacks 18 files (3 GB) to the chat in 14 s; 0 bytes received or sent over the network; 0 crashes; PSS 3.09 GB after 5 answers; a child ibuprofen-dose question gets the fixed health text; the first answer took about 2 min (model load), later answers 5.9-7.4 s at 15.9-16.0 tok/s. unpack 14 s; network rx 0 B / tx 0 B; PSS 3.09 GB; 0 crashes · set device smoke (hand questions, Pixel 8a, airplane mode) (dev, development data) · judge none (hand check) · n 5 questions · seeds UNKNOWN · CI not computed · date 2026-10-24 18:45 · caveat: One device, 5 hand questions. Findings: a unit conversion listed unrelated sources (fixed later by QuickMath); no loading indicator for the first answer (fixed later by the model preload); retrieval noise on one answer. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2200] QuickMath (43b50410): pure unit conversions and arithmetic are computed in code with no model, search, sources or line; health-gated turns and turns after a health turn are never computed. Model preload starts right after unpack (b8df70d9). The probe gate stays at 42 of 65 fixed text. 42 / 65 probes fixed text (unchanged) · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 18:54
- [F-2201] Device run of the candidate APK (5b457fc7: onboarding + an exposure rule) on the Pixel 8a: tour at 12 s after launch (unpack done), model ready at 18 s, first answer 11.9 s at 14.9 tok/s (the earlier ~2 min first answer is gone); "40 C in F" computed with the formula and no sources; 0 bytes over the network; 0 crashes. tour 12 s; model ready 18 s; first answer 11.9 s at 14.9 tok/s · set device smoke (hand questions, Pixel 8a, airplane mode) (dev, development data) · judge none (hand check) · n hand run (4 checks) · seeds UNKNOWN · CI not computed · date 2026-10-24 19:12 · caveat: One device, hand run. A cold-exposure question (stranded in a car at -20 °C) got the right Fahrenheit value with a wrongly stated formula and no cold-safety advice (medium finding); the last tour button wraps to two lines (low). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2206] BoarLite ships with the Wikipedia-health limitation: the Wikipedia packs stay in the kit (dropping them cost delivered facts 42 → 33 and topics 25 → 11), and the gate counts only the first-aid, MedlinePlus and health Q&A packs as health sources, so 12 of 60 health self-tests whose article is only in a Wikipedia pack get the fixed text; a curated list of medical Wikipedia titles is a post-submission item, re-gated on the 60 self-tests and 65 probes with the grounded-dose rule plus a check that toxic thresholds are not presented as doses. ship as is; medical Wikipedia titles after the submission · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 19:32 · caveat: who: coordinator, delegated by the author; reversible: yes. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2207] SEARCHPORT Addendum 2, ON_NO_HEALTH (the port on except on turns the health predicate reads as health): delivery MAIN 26 → 40, own fact chunk 38 → 62, 0 health losses, context 1.07×; S1 (65 probes with follow-up history): the port is off on 53, no probe changes its final path (41 fixed text, 23 model, 1 exposure both ways), 10 kept lists change, and ungrounded dose figures rise from 4 to 6 (a temperature on a carbon-monoxide probe, a duration that changed from hours to days, one repeated temperature): S1 FAIL; S2 (UXMATRIX 60, pack on) passed its frozen usefulness bar; frozen verdict: not WORTH-IT. MAIN 26 → 40; ST 38 → 62; ungrounded 6 vs 4 (grounded 9 vs 5); S2 bar passed · set MAIN 92 keyed; 180 pack self-tests; 65 F3-SAFETY probes (dev); UXMATRIX 60 (dev, development data) · judge replay (0 LLM); grounded-dose scorer; Claude Haiku labels for S2 (168 calls) · n 92 keyed MAIN; 180 self-tests; 65 probes; UXMATRIX 60 · seeds 1 (42) · CI not computed · date 2026-10-24 19:34 · caveat: S2 bar (frozen): correct-with-source +3 items or more, correct-any not lower by more than 1, wrong-unwarned not higher by more than 1. The S2 counts are BoarLite answer-quality numbers, published by the author's decision of 2026-10-25 (AD-15 (b); F-2208). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2209] The search port was merged into BoarLite as ON_NO_HEALTH although its frozen criterion failed by the letter (S1: 6 vs 4 ungrounded figures): before the run the author had set a standing condition ('I accept the health loss if it is worth it for the rest'), which the S2 result met. merged (mode ON_NO_HEALTH) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-24 19:35 · caveat: who: coordinator under the author's prior condition; frozen verdict: not WORTH-IT; reversible: yes (one switch). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2210] BoarLite final candidate b4ef94f3 (search port ON_NO_HEALTH): 0-LLM gate on the 65 probes with history searched as the app does gives 41 fixed text, 1 exposure answer and 23 model answers, identical with the port off (the port changes no health path); the invented-dose and blood-pressure probes stay fixed; the health predicate is byte-identical (0 new false fires); release APK 3,044,169,517 B with no INTERNET permission. 41 fixed / 1 exposure / 23 model of 65; APK 3,044,169,517 B · set 65 F3-SAFETY probes (dev) (dev, development data) · judge none (0 LLM) · n 65 probes · seeds UNKNOWN · CI not computed · date 2026-10-24 19:40 · caveat: Engine tests, device-test compile and debug build pass. Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2211] BoarLite candidate contents since the first release build: health gate (B+FU detector, after retrieval, exposure rule), the fixed line 'May contain errors; check the source.' under every model answer (replacing the ported warning line), factbook v1, QuickMath, model preload, a 5-card onboarding, exposure and conversion questions answered by code with a first-aid excerpt, search port ON_NO_HEALTH; open at the build session's close: phone smoke of the final APK, APK hosting (above a 2 GB asset limit), release-key backup, About text. · date 2026-10-24 19:44
- [F-2212] REFS-V2, OpenAI gpt-5.6-sol half complete: 75 of 75 v0.4 references written with live web search under the frozen protocol, 0 hard failures, 97 calls (two items completed on a resumed run); references of both judge families now exist. 75 / 75; 0 hard failures · set v0.4 (held-out ruler) (held-out ruler (exposed)) · judge UNKNOWN · n 75 items (88 turns) · seeds UNKNOWN · CI not computed · date 2026-10-24 20:05 · caveat: Protocol frozen 2026-10-21 before any call (prompt 5e41177d, generator eb2fd7e2). Reference text not read by the coordinator.
- [F-2214] Final ruler run (RULER-FINAL) on BoarLite's final engine b4ef94f3 exists as of 2026-10-24 20:34: desktop twin of the app path (QuickMath, search port ON_NO_HEALTH, health gate, history as the app sends it), 71 headline items + 4 anchors of bench v0.4, one seed, each judge family against its own references (Claude Opus vs the Claude references; OpenAI gpt-5.6-sol vs its references), 150 + 150 judge calls, addendum frozen before any generation; its numbers were held until the author's publication decision of 2026-10-25 (now public as F-2215 and F-2217; history, the headline is DEVICE-FINAL) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; now exposed to three engines)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · date 2026-10-24 20:34 · caveat: Deviations from the spec: one seed (spec: two; the author's choice), one reference family per judge family, stage 1 (DECLINE2STAGE) not run, no errata, desktop twin not the APK on a phone (the spec labels desktop-twin numbers dev/parity evidence). v0.4 is the claim set: no tuning after this run.
- [F-2215] RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 items. Claude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0 (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol (each vs its own references) · n 71 headline items · seeds 1 (42) · CI Claude 0.123-0.208; gpt-5.6-sol 0.194-0.300; min 0.123-0.206 · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).
- [F-2217] RULER-FINAL information-available split: Claude family U-total 0.164 where the installed packs hold the answer (n 19), 0.174 partly (n 25), 0.155 not at all (n 27); gpt-5.6-sol family 0.272 / 0.238 / 0.233: with the information present, the 1.7B does not deliver it. AVAILABLE 0.164 vs NONE 0.155 (Claude) (ruler U-total) · set v0.4 (held-out ruler; exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol · n 19 / 25 / 27 headline items · seeds 1 (42) · CI not computed · date 2026-10-24 20:34 · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Pre-registered stratifier (spec r4). Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup (F-2223, F-2225); not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).
- [F-2223] Between the ruler engine b4ef94f3 and LabRat 7788924a the engine module changed in 124 files (643 lines added, 97,537 removed), almost all removals of code and test resources described as unreachable in the closed app; the frozen DEVICE-FINAL criterion describes 7788924a's engine as b4ef94f3 plus the Rat persona line. Behaviour checks after the cleanup: 274 engine unit tests and a 5-question device smoke whose retrieved titles, health gate and conversion match the baseline; no ruler rerun. 124 files; +643 / −97,537 lines · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: Line counts from a read-only git diff of the engine module; the smoke compares titles, gate and conversion, not answer quality.
- [F-2224] BoarLite renamed LabRat at the app maintainer's request (commit 7788924a): the name changes in the UI, docs and assets; the assistant is called Rat (answer byline, notification, and the grounding prompt's identity line); a new rat mark in the app, launcher and splash. Unchanged: the package id, the code packages, the bundled knowledge catalogue and the About credit "Based on BOAR (MIT)". Phone smoke 6/6: titles, health gate and conversion match the baseline; "Who are you?" answers "I am Rat…" BoarLite → LabRat; assistant Rat; smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: A rename plus one prompt line; a smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.
- [F-2225] Engine change after the ruler run: in LabRat 7788924a the grounding prompt's identity line changed from "You are BOAR, an offline AI research app running on this phone…" to "You are Rat, the assistant in LabRat, an offline AI research app running on this phone…". The final ruler run (RULER-FINAL, 2026-10-24 20:34) was on b4ef94f3, before this change: its number is for b4ef94f3 (desktop twin), and DEVICE-FINAL measures LabRat 7788924a on the phone. persona line changed after RULER-FINAL · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 02:20 · caveat: The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.
- [F-2227] DEVICE-FINAL criterion frozen (sha256 recorded) on 2026-10-25 03:3x, before any matrix generation: LabRat 7788924a answers on a Pixel 8a 8 GB in airplane mode on AC power. Matrix: M1 ruler v0.4 (75 items, 88 turns) on seeds 42, 43 and 44, judged as in RULER-FINAL (Claude Opus vs Claude references on all three seeds; OpenAI gpt-5.6-sol vs its references on seed 42, further seeds optional), reporting per family the U-total, CI, win-rate, refusals, the minimum over families, the three-seed mean and a phone-vs-desktop parity read against the b4ef94f3 desktop twin (not a verdict); M2 the 65 safety probes (0 LLM); M3 the 140 on-K13 pack self-tests and M4 the 60 UXMATRIX items (Claude Haiku labels); M5 latency, time to first token, decode speed, memory, battery temperature, three cold starts and the app's network counters over the night. Claim numbers = M1 (both families named, seeds stated, win-rate first) and M5 (speed, memory, offline); M2-M4 are product numbers. No tuning, no engine change. matrix M1-M5; claim numbers = M1 + M5 (ruler U-total) · set bench v0.4 (exposed); 65 safety probes; 140 on-K13 self-tests; UXMATRIX 60 (dev, development data) · judge Claude Opus + OpenAI gpt-5.6-sol (M1); Claude Haiku 5.5-low labels (M3, M4); none (M2, M5) · n 75 items × 3 seeds + 265 product items · seeds 42, 43, 44 (M1); 42 (M2-M4) · CI not computed · date 2026-10-25 03:3x · caveat: No DEVICE-FINAL result is in this record; the matrix was running as of this update.
- [F-2230] APK hosting decided: the 3.04 GB LabRat APK goes to Hugging Face (author's OK on 2026-10-25) because GitHub release assets must be under 2 GiB; a public install page is planned with the download link, sha256, about 6 GB free space, Android 8+, "install unknown apps", the Play Protect prompt and a first launch of about 12 s. The code repository stays private unless the author decides otherwise. Hugging Face; upload pending · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: As of this update the upload is not recorded as done. · later: 2026-10-25 15:3x: the APK is hosted on Hugging Face (huggingface.co/r4topunk/labrat); the code repository is public (labrat-boar-fork).
- [F-2231] Install-path limits found before release: Android developer verification is live since 2026-09-30 in Brazil, Indonesia, Singapore and Thailand; adb installs are exempt, but browser installs of apps from unregistered developers may hit an "advanced flow" (a warning and a one-day wait), scope unconfirmed. The test phone runs GrapheneOS, which is not Google-certified, so it cannot prove the judges' install path; a browser-install test on a stock certified phone is planned, and registering the app with the developer verification is the mitigation (needs the author's Google account) risk open · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: Policy scope as checked by the build session on 2026-10-25, not confirmed on a certified phone.
- [F-2232] Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat shows. open; fix needs a ruler rerun · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: No fix is in the record as of this update.
- [F-2235] Latest LabRat APK (0ad476ca, bundled, same signing identity, no INTERNET permission) installed on the Pixel 8a after the rename and logo: smoke 6/6 matching the baseline (retrieved titles, health gate and conversion); the earlier cleanup build ff50c5ff is 3.04 GB. The APK built from b4ef94f3, the ruler engine, is kept. smoke 6/6 · set device smoke (dev) (dev, development data) · judge none (smoke checks) · n 6 smoke questions · seeds UNKNOWN · CI not computed · date 2026-10-25 11:24 · caveat: A smoke test, not a quality measurement. The final ruler number (RULER-FINAL) measures engine b4ef94f3 on its desktop twin; LabRat 7788924a adds the Rat persona line to the grounding prompt after that run, so the ruler number is not a measurement of LabRat 7788924a.
- [F-2236] The author restated the thesis on 2026-10-25: small models need external knowledge; the project learned that and built a harness around a small model's real capabilities, tuned for the app user's experience (health gate, QuickMath, pack search and delivery, the Succinct persona, a fixed check-the-source line, refusal behaviour). It replaces the 2026-10-23 sentence "right whenever the content is in a pack" as the thesis statement; the conditional closing-test numbers are its evidence and its limits. thesis statement (author, 2026-10-25) · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: who: author; a framing statement, not a measurement. Evidence: with the matching pack the 1.7B is right on 48 % vs 26 % of pack-written self-tests (F-1704); 65 % when the item's own article reaches the model vs 28 % (F-1705). Limits: the article is delivered on 54 % (F-1706); no pack reached the 0.75 bar (F-1707); no gain on real development items (F-2080). Never "the harness makes it half as good".
- [F-2237] Author publication decisions of 2026-10-25: numbers of the author's own submission (BoarLite, renamed LabRat) are publishable, numbers of the app maintainer's app stay internal; the headline is the final v0.4 ruler measured on the phone (DEVICE-FINAL); the desktop ruler run of b4ef94f3 is history with its code-difference note; the citation-support gap (Citations.applyShown) is documented as a known limitation and not fixed before the measurement. LabRat numbers public; headline = DEVICE-FINAL; applyShown not fixed · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 · caveat: who: author. Not fixing applyShown keeps the measured engine identical to the app judges install; the app maintainer's app numbers need his consent.
- [F-2238] DEVICE-FINAL M1, the final ruler measured on the phone: LabRat on a Pixel 8a (8 GB, airplane mode) answered bench v0.4 (71 headline items + 4 anchors) on seeds 42 and 43. It won 0 of 71 head-to-heads under both judge families (one tie under OpenAI gpt-5.6-sol on seed 42). U-total, both seeds pooled: Claude Opus 0.165 (0.127–0.205), OpenAI gpt-5.6-sol 0.242 (0.194–0.296). Per seed: Claude 0.167 (0.129–0.209) and 0.162 (0.125–0.201); gpt-5.6-sol 0.244 (0.193–0.298) and 0.241 (0.190–0.297). The lower family, Claude at 0.165 (0.127–0.205), is below the 0.50 bar. 5 headline items were refusals; answer coverage 0.90. 0 of 71 wins under both families; U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol); below the 0.50 bar (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items + 4 anchors · seeds 42, 43 · CI Claude 0.127-0.205; gpt-5.6-sol 0.194-0.296 (pooled seeds 42+43) · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. Two seeds (42, 43): the criterion listed 42, 43 and 44; seed 44 was dropped for time (two seeds = the ruler spec). Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.
- [F-2239] DEVICE-FINAL phone vs desktop twin (seed 42, paired over the 71 headline items): Claude 0.167 on the phone vs 0.164 for the desktop twin of b4ef94f3, +0.003 (−0.022…+0.026), 21 items better and 16 worse; gpt-5.6-sol 0.244 vs 0.245, −0.007 (−0.039…+0.023), 22 better and 22 worse. The phone repeats the desktop number; a parity read, not a verdict. paired +0.003 (Claude) / −0.007 (gpt-5.6-sol) (ruler U-total) · set bench v0.4 (exposed) (held-out ruler (v0.4; exposed)) · judge Claude Opus + OpenAI gpt-5.6-sol via the Codex CLI (each vs its own references) · n 71 headline items · seeds 42 · CI Claude −0.022…+0.026; gpt-5.6-sol −0.039…+0.023 · date 2026-10-25 13:2x · caveat: Metric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. v0.4 has been answered by our engines since 2026-10-24, so it is no longer unseen (F-2214). One reference family per judge family; stage 1 not run; no errata. The two engines differ by the Rat persona line and platform sampling. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.
- [F-2240] DEVICE-FINAL M5, speed, memory and offline on the Pixel 8a (model-written turns of v0.4 seed 42, n 82): total answer time p50 13.9 s (p90 19.6 s), first word p50 9.8 s (p90 14.5 s), decode 11.8 tokens/s, search 1.6 s, peak memory (RSS) 3.87 GB, battery at most 36.4 °C. After 45 minutes of continuous use (seed 43): 17.0 s p50 and 8.9 tokens/s. Cold starts: model load 1.6–1.7 s, first word 10.4–19.4 s. A fresh install on a cool phone decoded 16.7 tokens/s. Network: the app has no traffic record in the phone's network statistics before or after the matrix, and no internet permission. p50 13.9 s; first word 9.8 s; 11.8 tok/s; 3.87 GB; 0 network traffic (ruler U-total) · set bench v0.4 turns on the phone (dev, development data) · judge none (device metrics) · n 82 model-written turns per seed · seeds 42 (43 for the warm read) · CI not computed · date 2026-10-25 13:2x · caveat: One phone (Pixel 8a, GrapheneOS, AC power, screen on). Heat roughly halves decode speed under sustained use: quote the range. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.
- [F-2241] DEVICE-FINAL M2, the 65 written health and safety probes on the phone (0 LLM scoring): 53 got the fixed safe health text and 12 went to the model (the desktop twin of b4ef94f3: 41 fixed, 1 exposure answer, 23 model); 2 model answers added a dose figure the question did not ask for. One of them (a carbon-monoxide reading of 200 ppm at home) calls the level unsafe but understates an emergency; the author kept it as a known limitation. 53 of 65 fixed text; 2 new dose figures · set 65 written safety probes (dev; the gate's rules were tuned on them) (dev, development data) · judge none (0 LLM scorer) · n 65 probes · seeds 42 · CI not computed · date 2026-10-25 13:2x · caveat: Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice. Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.
- [F-2242] DEVICE-FINAL M3, 140 pack-written self-tests of the 7 tested packs that LabRat ships (20 each) on the phone: correct-any 0.400 (0.314–0.479); 0.476 when the question's own article was kept (n 82) vs 0.293 when it was not (n 58); wrong-any 0.436. correct 0.40 of 140 (outcome-label share) · set pack-written self-tests (dev) (dev (pack-written self-tests), development data) · judge Claude Haiku 5.5-low labels (UXMATRIX protocol) · n 140 · seeds 42 · CI 0.314-0.479 · date 2026-10-25 13:2x · caveat: Outcome-label shares, not the ruler scale. Self-tests favour the packs. Correct-with-source is a lower bound (phone rows hold source titles only). Health content: AI-reviewed, not clinically reviewed. Nothing here is medical advice.
- [F-2244] DEVICE-FINAL ran as one interrupted and resumed run: seed 42 stopped at 77 of 88 turns at 11:31 when an APK install of LabRat 0ad476ca (brand files only) reached the phone; the 69 complete items were kept and the 6 unfinished ones were re-run on 0ad476ca; seeds 43 and the M2-M5 matrix ran entirely on 0ad476ca; seed 44 was dropped for time. interrupted once, resumed; seed 44 dropped · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 11:31 · caveat: Process fact.
- [F-2246] LabRat 1.0.0 released: APK dcc515fc, 3,041,570,902 bytes, sha256 d2e00ba4a9adff6f903eab7d34759fd4f4db9cd721303dc7c38ae280b465dae8, hosted on Hugging Face at huggingface.co/r4topunk/labrat and installed on the Pixel 8a. The bundle was rebuilt from public pinned URLs only (16 of 16 inputs fetched and checked by sha256) 1.0.0 hosted; sha256 d2e00ba4… · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 15:3x · caveat: Measured build: LabRat 0ad476ca (engine = 7788924a, b4ef94f3 plus the Rat persona line; 0ad476ca changes brand files only). The released LabRat 1.0.0 (dcc515fc) has the same engine module (empty git diff); its later changes are UI, credits, docs and build scripts.
- [F-2247] The LabRat source is public at github.com/r4topunk/labrat-boar-fork (MIT, a fork of BOAR), with README, results, pack provenance and licence; made public with the author's OK. public repository · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25 15:4x
- [F-2248] Knowledge-pack counts, one meaning each: LabRat ships 13 packs (the K13 kit), and every one of the 13 exact files is downloadable from the boar-packs dataset on Hugging Face (7 of them under labrat-k13/: 6 earlier packs with added chunk vectors and 1 new pack). The 2026-10-23 self-tests covered 9 packs (180 questions), 7 of them in the app; DEVICE-FINAL tested those 7 on the phone (140 questions) 13 shipped; 9 self-tested; 7 tested on the phone · set UNKNOWN (status UNKNOWN) · judge UNKNOWN · n UNKNOWN · seeds UNKNOWN · CI not computed · date 2026-10-25

<!-- lint:exempt:end -->
---

Cite as: r4to, independent researcher. RatLab — A Boar Research (2026), research record 1.0, as of 2026-10-25. Text and data CC BY 4.0. Fact ids or a link must travel with a quote. How to cite: https://r4topunk.github.io/boar-research/cite/ · https://r4topunk.github.io/boar-research/CITATION.cff.

AI-reviewed, not clinically reviewed. Nothing here is medical advice.