RatLabA Boar Research
Closed recordas of what’s still open →

The research · Act II, Pushing the engine · Chapter 06

The deep tier

The one lever that beat noise was a much bigger model, a 35B mixture-of-experts with only about 3B active per token, and letting it think for 512 tokens before answering; it climbs the ladder from 0.26 to 0.44 floor-adjusted on the bounty-style set (desktop only, dev-style data), and still wins no head-to-head.

Full thesis

It never ran on a phone: on 2026-10-21 the author decided that no 35B deep mode is delivered for the bounty; the deep tier stays in the record as a roadmap with its desktop numbers (open fronts “Deep tier on a 12 GB-class phone” and “Expert-streaming engine on Android” dropped) 35B deep mode not delivered; deep tier appears only as roadmap with desktop numbersstoodDate2026-10-21Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: author (GRILL); why: focus on 8 GB UX; no Android expert-streaming build; alternatives: 12 GB deep tier; reversible: yesSourcenot in the public repositories yetOpen the data file.

2026-10-02, 10-06 → 10-10; dropped from delivery 10-21← In the article: §4 The cemetery of ideas

The thinking budget · desktop only

Thinking pays up to 512 tokens; after that it only costs time

The 35B model with a cap on how long it may think before answering. Top: quality, floor-adjusted. Bottom: median seconds per answer on a desktop computer.

Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set.raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 sSetV-paraStatusdev-styleJudgeClaude Opus (judge.v1, both orders)n51Seeds2Date2026-10-09CaveatChosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only.Sourceresearch record reports/BAR-TABLE.md:39research record reports/BAR-TABLE.md:126research record README.md:28Open the data fileA 512-token thinking budget matches 1024 in quality at about half the time.512 vs 1024 on V-para: raw +0.011 (−0.010…+0.031), floor-adjusted +0.014 (−0.012…+0.039), ×0.55 time; on v0.2 −0.005 (−0.020…+0.009), ×0.62 timeSetV-para; v0.2Statusdev-style / devJudgeClaude Opus (judge.v1, both orders)n51 / 110Seeds2 (V-para) / 1 (v0.2)Date2026-10-09CaveatVerdict label: CHEAPER-PARITY (an equivalence, not a gain).Sourceresearch record reports/CLOSED.md:39research record reports/BAR-TABLE.md:107research record reports/CLOSED.md#R20Open the data fileThinking budget 3072 added nothing: the model stops by itself at ~1.3k tokensΔ +0.010 (−0.014…+0.041)SetV-para dev-styleStatusdev-styleJudgeHaiku L1nnot recordedSeeds1Date2026-10-08Sourceresearch record reports/CLOSED.md:41Open the data file
Data table
budget (tokens)floor-adjusted (95 % CI)median s per answer
00.329 (0.294–0.362)33
2560.387 (0.349–0.426)55
5120.436 (0.401–0.472)79
10240.422 (0.385–0.459)143
3072no further gain (+0.010 vs 1024, 1 seed)not timed

A much bigger brain that mostly sleeps

The one idea that clearly beat the noise was a much bigger model: 35 billion parameters, built as a "mixture of experts". Think of a hospital with many specialists where each patient only sees a few. Only about 3 billion parameters work on each word, so it can run on strong hardware without needing all of its weights at once.

Scratch paper with a page limit

Then we let it think before answering: it writes hidden notes, up to a budget, and only then the answer. Thinking was the biggest single step we measured. And the budget had a sweet spot: 512 tokens of notes (a token is roughly a word piece) were as good as 1,024, in about half the time.

Climbing the ladder

On the bounty-style questions, the honest score climbs: 0.261 for the phone's small model, 0.329 for the big model, 0.436 for the big model with thinking. The line is 0.50.

V-para · practice-style for the big model · Claude Opus · 51 questions · 2 runs

Still zero wins

Even at its best, the big model did not beat the frontier answer on a single one of the 51 questions. It would also need a phone with 12 GB of memory and, by our estimate, two to three minutes per answer. It only ever ran on a desktop: on 21 October the author decided not to ship it, and it stays in this record as a possible future, with its desktop numbers.

What's still missing

When we looked at where its remaining points were lost, most of it was knowledge the phone does not hold and the model does not know, and the rest was confident wrong facts.

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

The deep tier

35B mixture-of-experts + 512 tokens of thinking: 0.261 → 0.436 floor-adjusted, 79 s per answer on desktop; never delivered

The only lever that beat noise on bounty-style questions was a larger model, and it stayed on the desktop: on 2026-10-21 the author decided that no 35B deep mode is delivered. Qwen3.6-35B-A3B activates about 3B parameters per token, so it fits a desktop CPU at 2-bit (UD-Q2_K_XL). On V-para (Opus, two seeds; dev-style data, desktop only) the ladder is: 4B claim build floor-adjusted 0.261 → deep tier 0.329 → deep + thinking 1024 0.422 → deep + thinking 512 0.436, all 0 wins of 51. Thinking is the biggest step (+0.093 floor-adjusted, CI +0.050…+0.137), and the budget has a knee: 512 tokens matches 1024 (+0.014 floor-adjusted, CI −0.012…+0.039) at 0.55× the time. Desktop p50 per answer: 4B ≈ 6–10 s, deep ≈ 31–33 s, deep + 512 ≈ 79 s, deep + 1024 ≈ 143–146 s.

config raw (CI) floor-adj (CI) W/T/L desktop p50 status
4B claim build (cut 6 = cut 7) 0.413 (0.384–0.445) 0.261 (0.224–0.300) 0/0/51 6.3–10.1 s held-out at run time
deep, no thinking 0.467 (0.439–0.494) 0.329 (0.294–0.362) 0/0/51 30.9–33.3 s dev-style
deep + thinking 256 0.514 (0.483–0.544) 0.387 (0.349–0.426) 0/0/51 53.2–57.5 s dev-style
deep + thinking 1024 0.542 (0.513–0.571) 0.422 (0.385–0.459) 0/0/51 142.9–146.0 s dev-style
deep + thinking 512 (recommended) 0.553 (0.524–0.581) 0.436 (0.401–0.472) 0/0/51 78.9–79.4 s dev-style; best of ≥ 8 configs

W/T/L is per seed and identical for both seeds on every row.

Gotchas

  • The on-device MoE engine (bmoe v0.28.0) has no thinking-budget flag; a phone build needs an app-side cap bmoe v0.28.0 has no thinking-budget flag; a device needs an app-side cap.Date2026-10-09Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatunbounded thinking ≈ 1.3k tokens, ≈ 2× timeSourcenot in the public repositories yetOpen the data file.
  • The deep tier still needs retrieval: removing packs costs −0.045 (−0.083…−0.011) (slice of 20, 1 seed, Haiku) Retrieval still matters for the deep tier (removing packs hurts)−0.045 (−0.083…−0.011)SetV-para slice devStatusdevJudgeHaiku N=4n20 (slice)Seeds1Date2026-10-10Caveatslice power lowSourceresearch record reports/CLOSED.md:52research record reports/CLOSED.md#R33Open the data file.
  • Bigger thinking budgets do nothing: the model stops by itself around 1.3k tokens Thinking budget 3072 added nothing: the model stops by itself at ~1.3k tokensΔ +0.010 (−0.014…+0.041)SetV-para dev-styleStatusdev-styleJudgeHaiku L1nnot recordedSeeds1Date2026-10-08Sourceresearch record reports/CLOSED.md:41Open the data file.

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

6.2 Model scale and test-time compute

All deep-tier rows were measured on V-para after it became development-style and on v0.2 after it was burnt; they are development evidence, desktop only. On V-para (Opus, two seeds): deep without thinking raw 0.467 (0.439–0.494), floor-adjusted 0.329 (0.294–0.362); with a 1024-token thinking budget 0.542 / 0.422 (0.385–0.459); with 512 tokens 0.553 (0.524–0.581) / 0.436 (0.401–0.472), common-mode floor-adjusted CI 0.387–0.486; with 256 tokens 0.514 / 0.387 (0.349–0.426). Every row wins 0 of 51 in each seed. Thinking (1024 vs none) adds +0.093 (+0.050…+0.137) floor-adjusted, the largest single effect we measured; 512 vs 1024 is parity (+0.014, −0.012…+0.039) at 0.55× the wall time; 256 vs 1024 loses −0.028 raw (−0.056…−0.003); a 3072 budget adds nothing (the model stops near 1.3k tokens; 1 seed, Haiku). The 512 configuration was selected as the best of at least eight deep configurations on V-para, so a winner's-curse discount of ≈ 0.01–0.02 applies; over three seeds its floor-adjusted mean is 0.432 (SD 0.025). On v0.2 the 512 configuration scores raw 0.608 (0.575–0.641), floor-adjusted 0.508 (0.467–0.550), W/T/L 0/1/109 with one seed; the 1024 configuration with two seeds scores 0.612 (0.581–0.644) / 0.514 (0.475–0.554).

6.3 Right but thin: dimension shares, reference length and the loss map

Decomposed by rubric dimension, the 512 configuration on V-para is above 0.50 floor-adjusted on correctness alone (0.660, 0.608–0.712) and well below it on completeness (0.301) and usefulness (0.341); the 4B is 0.427 on correctness. Re-judging the same answers against references condensed to ≤ 80 words moves the 512 configuration from 0.412 to 0.514 (0.456–0.578) floor-adjusted (seed 42, Opus), a paired +0.102 (+0.068…+0.144); half of that shift comes from the shortened references scoring lower themselves, and we attribute ≈ 17 % of the gap to reference length and ≈ 83 % to substance. This is not a bar reading: references were altered and n is one seed. A loss map attributes 59 % of the remaining V-para loss to knowledge absent from both the kit and the model and 40 % to wrong facts; numbers and dates dominate wrong-fact loss. All of English Wikipedia (Aug-2025 + a 2026-10-01 delta) verifiably holds the missing fact for 9 of 34 absent bounty-style items, a lower bound.

Badge rows the 6 public fact rows this depth cites; a number here without a chip comes from a row that is not public yet (see /data/)
Claim and valueSetStatusJudgenSeeds95 % CIDate
Swapping in the 35B mixture-of-experts deep tier (no thinking) raises the bounty-style score.raw 0.467 (0.439–0.494); floor-adjusted 0.329 (0.294–0.362); win 0/51; desktop p50 ≈ 33 sV-paradev-style (measured after 2026-10-06)Claude Opus (judge.v1, both orders)5122026-10-08
Thinking budget 3072 added nothing: the model stops by itself at ~1.3k tokensΔ +0.010 (−0.014…+0.041)V-para dev-styledev-styleHaiku L1not recorded12026-10-08
On correctness alone H is above 0.50 floor-adj; the gap is in completeness and usefulnessV-para correctness floor-adj 0.660 (0.608–0.712) vs completeness 0.301 / usefulness 0.341V-para devdevOpus51 (inferred from set)22026-10-10
Where the deep tier loses points (LOSSMAP)absent knowledge 0.59 (V-para) / 0.55 (v0.2) of loss; wrong fact 0.40 / 0.21V-para dev-style + v0.2 devdev-styleOpus rationales classified by jev (a calibrated classifier)not recorded12026-10-11 12:10
Numbers and dates dominate the wrong-fact loss; declining instead of a wrong answer would lower the score (RATIONALEPASS)numeric + date 0.51 (V-para) / 0.70 (v0.2) of wrong loss; decline oracle −0.064 / −0.040V-para / v0.2not recordedjev over Opus rationalesnot recordednot recorded2026-10-12 14:17
Wikipedia-anywhere gate for V-para absent-knowledge items9/34 verified (MARGINAL band 7–13)V-para devdevjev + quote check51 (inferred from set)not recorded2026-10-12/13

In one sentence

A bigger model that thinks first got closest, 0.436, on a desktop, with no wins, and it was never shipped.

Turning point

2026-10-08 20:20, DEEPTHINK: thinking adds +0.093 floor-adjusted, the biggest step on the ladder.

On the calendar

31 dated events · 2 Oct → 19 Oct

  1. 2 OctFirst deep-tier test: Qwen3.6-35B-A3B +0.060 over the 4B (first lever above MDE)
  2. 2 Oct · 12:55Deep tier (35B MoE) approved; counts only if reported separately from the default
  3. 6 OctDirectional day: DEEPREAD, COMPLETE, NAMEDSLOT, FACTS; "reader-capability gap"
  4. 7 Oct · 00:11MOESCREEN: 25–35B MoE closed-book plateau ≈ 0.60 vs 4B 0.487
  5. 8 OctDEEPVPARA: deep tier on V-para floor-adj 0.329 vs 4B 0.261 (two seeds, Opus)
All 31 events
  1. 2026-10-02First deep-tier test: Qwen3.6-35B-A3B +0.060 over the 4B (first lever above MDE)
  2. 2026-10-02 12:55Deep tier (35B MoE) approved; counts only if reported separately from the default
  3. 2026-10-03Token budget: work pauses; deep tier paused at D1
  4. 2026-10-04Direction: 4B default tier only; deep tier parked; no team chat agents, coordinator + subagents
  5. 2026-10-06Directional day: DEEPREAD, COMPLETE, NAMEDSLOT, FACTS; "reader-capability gap"
  6. 2026-10-06DEEPREAD: deep model reads gold passages like the 4B (+0.029, CI −0.004…+0.063) → its gain is knowledge
  7. 2026-10-07 00:11MOESCREEN: 25–35B MoE closed-book plateau ≈ 0.60 vs 4B 0.487
  8. 2026-10-08DEEPVPARA: deep tier on V-para floor-adj 0.329 vs 4B 0.261 (two seeds, Opus)
  9. 2026-10-08Thinking mode on the deep tier: V-para floor-adj 0.422
  10. 2026-10-08Opus judge sharding for claim numbers; 20-item slice before any full deep-tier run
  11. 2026-10-08SLICE20: every deep-tier / held-out-style arm starts on a 20-item slice
  12. 2026-10-08Deep tier thinking budget curve; 512 adopted (DT512)
  13. 2026-10-08 20:20DEEPTHINK: deep + thinking 1024 = 0.422 floor-adj; +0.093 over no thinking
  14. 2026-10-09Gemma 4 26B + thinking: faster, not at parity → NO
  15. 2026-10-09v0.2 deep + thinking two seeds: floor-adj 0.514 (CI straddles 0.50), win 0/110
  16. 2026-10-09 02:32DEEPTHINK-V02 L1 NO at Haiku +0.027 (later Opus addendum: v0.2 floor-adj 0.514)
  17. 2026-10-09 20:45DT256: budget 256 loses −0.028 vs 1024 (CHEAPER-LOSS); knee at 512
  18. 2026-10-09 23:14DT512-V02: H on v0.2 floor-adj 0.508 (0.467–0.550), first point above 0.50, CI straddles
  19. 2026-10-10Best-of-2, sampling presets, context size, corpus size, thinking 4B: all NO
  20. 2026-10-10THINK4B: thinking twin of the 4B +0.026 (CI crosses 0), ×3.7 time → NO
  21. 2026-10-10Desktop queue empty: 4B side and deep-tier line both closed
  22. 2026-10-10 22:32DEEPRERANK, MP4B, DEEPITER, DEEPCOV-FB closed (retrieval-side levers on the deep tier)
  23. 2026-10-11 12:10Loop tick 1: LOSSMAP measures where the deep tier loses points
  24. 2026-10-12 00:34DEEPCOVER NO (correctness −0.11), VITALADD NO at slice
  25. 2026-10-12 01:28QUANTCEIL: 3-bit quant of deep model +0.004 (not detected); cycle adds 0 V-para points per 1,804 judge calls
  26. 2026-10-12 08:07SYNTHDEEP, WIKISECTION, DEEPHEDGE closed at 0-call gates
  27. 2026-10-12 08:21Sequencing incident: the 35B Q2 model was deleted for QUANTCEIL while DEEPHEDGE needed it
  28. 2026-10-12 08:21Model swap for one arm deletes the 35B model another arm still needs (second time) → rule 22
  29. 2026-10-14 17:06RIVALSET-DEEP: deep tier 3.92 on the competitor port, gap is obscure-subject coverage
  30. 2026-10-16 10:05Lite line closed: readers, quant, reach budget, follow-up, query rewrite, thinking
  31. 2026-10-19 19:0xLESANDROID: expert-streaming engine cross-built for Android arm64, phone unmeasured

Related nodes on the map

This chapter on the map →

Concepts

Experiments

Decisions

Key findings

.md