The map · 164 nodes · 109 links
Everything, connected
The story runs clockwise around the ring, one chapter per hub. Around each hub sit the ideas, tests and choices of that chapter; lines show what led to what, what was ruled out, and what uses which idea. The author's thesis and the on-phone headline sit in the middle. Point at or tap a node to see its neighbours; open it to read its note. On a phone, the list below is the quickest way in.
Browse the map as a list, chapter by chapter
01The question
The goal was simple to say and hard to measure, so the first job was building the ruler.
02Building a ruler
Every answer is graded twice, blind, against a frozen frontier answer, under rules written before the test.
03The floor trap
We grade on the scale where a blank answer scores zero, and still show the flattering scale beside it.
04Seven frozen builds
Seven builds made the app safer and quicker, not smarter.
- cut (frozen build)concept
- emergency detector (DETMAX)concept
- FACTS tableconcept
- ARexperimentDoes a deterministic arithmetic calculator (rate x quantity -> time, ceil-division) plus a detector…
- CUT3DEVexperimentBuild the three 4B-LOSS-ANATOMY arms (VG, AR, SC) as commits on cut 2 and assemble the adoptable cut 3,…
- FACTS-r6experimentDoes asset coverage expansion (1,036 -> 4,233 entities) plus 5 trigger fixes on r5 reach >=85% recall on…
- NAMEDSLOTexperimentDoes giving a question-named low-share title one lead passage after the first hits (cut 6 + one…
- CUT7experimentDoes adding FACTS r6 to the frozen cut 6 build change any v0.2 held-out or V-para answer, and if so are…
05The walls
The small model reads well enough; what it lacks is knowledge the phone doesn't hold.
06The deep tier
A bigger model that thinks first got closest, 0.436, on a desktop, with no wins, and it was never shipped.
- deep tierconcept
- thinking budgetconcept
- Recommended deep configuration (35B…finding
- Stop answer-identical 4B armsdecision
- Recommend thinking budget 512 for the deep…decision
- Declare the deep-tier line closed on desktop…decision
- 35B deep mode not delivereddecision
- COMPLETEexperimentDoes a completeness instruction (120-220 words, max tokens 700) on the deep35 reader with gold R450…
- DEEPREADexperimentDoes the deep model (Qwen3.6-35B-A3B UD-Q2_K_XL) convert gold retrieved passages (R450 oracle contexts)…
- DEEPROUTEexperimentDescriptive (0 calls): where does the deep tier (Qwen3.6-35B-A3B engine, night-10-01) beat the 4B engine…
- DEEPTHINKexperimentDoes a 1024-token thinking budget on the deep tier (arm T: cut 6 + Qwen3.6-35B, thinking on) lift V-para…
- DT512experimentDoes halving the deep tier's thinking budget (DEEP_THINKING_BUDGET 1024 -> 512, arm H) keep the thinking…
- GEMMATHINKexperimentDoes Gemma 4 26B-A4B UD-Q2_K_XL + 1024-token thinking as the deep tier (arm G) match or beat…
- THINK4BexperimentDoes the thinking twin of the claim 4B (Qwen3-4B-Thinking-2507 Q4_K_M) on the claim build's fast path, at…
07A phone on a desk, and in a hand
The phone matches the desktop on quality; it is just slower, and the big model never left the desktop.
- desktop proxy and device parityconcept
- expert streaming (bmoe, LES)concept
- Cut 7 = cut 6 + FACTS r6 (adopted on gates: 0…decision
- Win-rate reported, never gated (laterdecision
- DEEPPAR-L1experimentD4 step (c): at the judged level, does bmoe v0.28.0 produce answers of the same quality as llama-server…
- VENUEGPS-2experimentWith a device GPS fix AND the user's area places tile installed (kit-vexact + coordinate-selected tiles),…
- DEVICE-X6experimentCut 7 on a 12 GB-class phone (DEVICE-X6, measurement; a POCO X6 Pro test phone, MT6897, 10.7 GiB)
- LESPORT-L1experimentCan an expert-streaming llama.cpp fork run the 35B deep model in a small RAM budget with…
08The agent lab
An AI-agent lab can run real research, if every rule is written down and every mistake becomes a new rule.
09Knowledge packs
Pick the one library you need; more is not better.
- 180 pack self-testsconcept
- crowdingconcept
- HEADPASS-X2bconcept
- knowledge packconcept
- Pivot to Knowledge Packsdecision
- Withdraw "facts are not in Wikipedia"decision
- HEADPASS-X2b = the pack line's retrieval…decision
- DEEPNORAGexperimentDo the knowledge packs still matter for the deep tier with thinking 512: arm N (DT512 config with…
- WIKIFULL-STAGE-AexperimentOver the 34 V-para absent-knowledge items (H s42), is the missing fact present anywhere in the 15…
- ORACLE-STexperimentRetrieval ceiling: does injecting the known fact chunk into the 4B cut 7 context (ORACLE-ADD extra slot;…
- SFT4BexperimentCan a small LoRA (rank 16, ~1,200 extraction-style grounded QA examples from the packs' own text,…
- HEADPASS-X2bexperimentDoes appending the retrieved article's neighbouring chunks plus up to 2 dense-vector candidates (≤ +600…
- HEADPASS-X3experimentCross-encoder reranker choosing the appended chunks on the HEADPASS path (X3a reranker-only, X3b X1 +…
- LITE5-180experimentPack retrieval reach for the lite tier (HEADPASS-X2b on Qwen3-1.7B Q4_K_M; LITE5 → LITE5-180)
10Measuring the measurer
A different judge changes the level, not the ranking, and not the wins.
- cross-family judge (XFAM)concept
- judge family / self-preferenceconcept
- seed and seed noiseconcept
- winner's curseconcept
- A judge from a different model family…finding
- HAIKUJUDGEexperimentDoes Claude Haiku 5.5 (effort low) agree with Opus judge.v1 at least as well as Sonnet-low on deep35-R450…
- LENGTHSHAREexperimentMeasurement: how much does the 3-dim floor-adjusted ratio of H (deep + thinking 512, s42) and K (4B cut…
- SEEDVARexperimentHow much do the recommended deep configuration's (DT512 arm H: cut 6 + Qwen3.6-35B + thinking 512) V-para…
- XFAMexperimentDoes a judge from another model family change the levels, ranking or win-rate of the 4B and deep-tier…
- XFAM-V02experimentDoes the cross-family offset hold on v0.2?
11A smaller engine for 8 GB phones
For an 8 GB phone, speed and memory picked the small model; it reads worse, and we say so.
12A second ruler (bench v0.4)
A fairer exam is written and sealed; the submitted app has sat it, and the grade waits for the author (as of 24 October 2026, 20:34).
- DECLINE2STAGEconcept
- FACT-AVAIL (information-available split)concept
- ruler (bench v0.4)concept
- U-totalconcept
- RULER-FINAL information-available split…finding
- RULER-FINAL U-total of BoarLite b4ef94f3…finding
- Build a public rulerdecision
- RULING R1: judge = judge.v1 unchanged +…decision
- Final ruler run frozendecision
- DECLINE2STAGEexperimentA Haiku stage-1 non-answer classifier before judge.v1
- ORACLE-REALexperimentDoes forcing the fact chunk to passage 1 help on real items?
- RULER-FINALexperimentWhat is BoarLite's U-total on v0.4 under both judge families?
13Delivery, not coverage
The phone usually has the book; the hard part is opening it at the right page.
- delivery (kept / delivered fact)concept
- K13 kitconcept
- SEFIX-124concept
- Pack self-test gain does not reproduce on…finding
- Correct-any when own article is kept vs…finding
- Own article delivered (kept)finding
- Packs reaching the 0.75 "proves" barfinding
- Delivery is measured at chunk level from now…decision
- Packs are the strategy: engine finds the…decision
- SEFIX-124experimentF1/F2/F4 without F3
- W78experimentCut phone reading cost of the dense stage
- X2B-BUDGETexperimentWider append budget
14Closing the research: what a small model needs
A small model needs a library: right about two times in three when the page arrives; the page arrives about half the time.
- conditional thesisconcept
- desktop twinconcept
- outcome labels (CWS, correct-any, WU, WW, HU)concept
- PACKPROOF pooled correct-any with pack…finding
- Thesis statement (author, 2026-10-25)finding
- D7: upload the research site only as v1.0 at…decision
- Close the research linesdecision
- CONTENT-FIRST: no model changedecision
- Thesis to present with the app: right whenever…decision
- Closed simple chat app for one way of use…decision
- Thesis restated: small models need external…decision
- NOWIKIexperimentKit without simplewiki and Vital
- PACK-GAPexperimentWould new packs close the undelivered gap?
- PACKPROOFexperimentWith vs without the pack on its own self-tests
- QREWRITEexperimentCeiling of question rewriting (oracle)
- UXMATRIXexperimentIn each mode, is the answer right when the pack holds the content, and what happens without it?
- VALIDATION judgeexperimentSuccinct vs S+ (one 4-5 sentence paragraph) vs two paragraphs
15One simple app: LabRat
The entry is a deliberately simple app, LabRat, that runs offline on a phone; as of 25 October 2026 its desktop dry run is below the bar and its final test on the phone, the one that counts, is below the bar too.
- DEVICE-FINALconcept
- fixed lineconcept
- health gate B+FUconcept
- LabRatconcept
- QuickMathconcept
- Final ruler (bench v0.4) measured on the phonefinding
- Open engine bug, as of 2026-10-25: the…finding
- No source found → refuse by default + "show…decision
- Submission = the maintainer's native Kotlin…decision
- BoarLite = a NEW bounty submission by the…decision
- BoarLite: minimal health pathdecision
- Search port merged into BoarLite as…decision
- Publish LabRat's numbers, keep the app…decision
- Rename BoarLite to LabRatdecision
- Run DEVICE-FINALdecision
- GROUNDED-DOSE AexperimentWith the widened health gate, does the port add ungrounded dose figures?
- SEARCHPORT Addendum 2 (ON-NH)experimentIs the port worth it for non-health questions if health turns keep the old search?
- SEARCHPORT P1experimentDoes porting SEFIX-124 + head pass + X2b make the right chunk reach the 1.7B without health or safety changes?
- DEVICE-FINALexperimentWhat are LabRat's claim numbers measured on the phone (ruler v0.4 under both judge families; speed,…
16Where it stands (live)
Not shown to be half as good: the desktop dry run of the new exam is below the bar, the phone run that counts is below the bar too, and a small model is right about two times in three when the page arrives (as of 25 October).
17Open fronts and hand-off
The research is closed; what remains is the phone test of the simple app, hosting it, and posting the claim.