Experiment · 2026-10-09

DT512

Does halving the deep tier's thinking budget (DEEP_THINKING_BUDGET 1024 -> 512, arm H) keep the thinking gain vs T (DEEPTHINK, 1024) while cutting time (with D = DEEPVPARA no-thinking for the ladder)?

Chapter 6 · The deep tierparity

From the record

What was tested: DT512
Result: 512 vs 1024 on V-para: raw +0.011 (−0.010…+0.031), floor-adjusted +0.014 (−0.012…+0.039), ×0.55 time; on v0.2 −0.005 (−0.020…+0.009), ×0.62 time
Verdict: parity: an equivalence (same quality, cheaper or transferable), not a gain
Detail: closed (CHEAPER-PARITY; 512 kept as a local engine recommendation, not in a claim cut)
Set: V-para (V-para); v0.1 (smoke, 2 items)
Judge: Claude Opus, Claude Haiku (L1)
n: 51 judged ids (58 generated) x 2 seeds; slice 20
Seeds: 42, 43 (third seed 44 noted later via SEEDVAR)
Pre-registered: yes, sha256-frozen
Reopen if: none: budget curve closed, knee 512

Linked facts · 3

A 512-token thinking budget matches 1024 in quality at about half the time.A 512-token thinking budget matches 1024 in quality at about half the time.512 vs 1024 on V-para: raw +0.011 (−0.010…+0.031), floor-adjusted +0.014 (−0.012…+0.039), ×0.55 time; on v0.2 −0.005 (−0.020…+0.009), ×0.62 timeSetV-para; v0.2Statusdev-style / devJudgeClaude Opus (judge.v1, both orders)n51 / 110Seeds2 (V-para) / 1 (v0.2)Date2026-10-09CaveatVerdict label: CHEAPER-PARITY (an equivalence, not a gain).Sourceresearch record reports/CLOSED.md:39research record reports/BAR-TABLE.md:107research record reports/CLOSED.md#R20Open the data file
Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set.Recommended deep configuration (35B mixture-of-experts + 512-token thinking budget) on the bounty-style set.raw 0.553 (0.524–0.581; with seed noise 0.513–0.592); floor-adjusted 0.436 (0.401–0.472; with seed noise 0.387–0.486); win 0/51; desktop p50 ≈ 79 sSetV-paraStatusdev-styleJudgeClaude Opus (judge.v1, both orders)n51Seeds2Date2026-10-09CaveatChosen as the best of ≥ 8 deep configurations on this same set: expect a winner's-curse discount of about 0.01–0.02. QUEUE prints raw 0.552; the two-seed aggregate is 0.553. Desktop only.Sourceresearch record reports/BAR-TABLE.md:39research record reports/BAR-TABLE.md:126research record README.md:28Open the data file
R20 Budget 512 vs 1024: 2 seedsR20 Budget 512 vs 1024: 2 seeds+0.011 (−0.010…+0.031), ×0.55 timeKinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatwhy closed: CHEAPER-PARITY → 512 adoptedSourceresearch record reports/CLOSED.md:39Open the data file