Experiment ·
THINK4B
Does the thinking twin of the claim 4B (Qwen3-4B-Thinking-2507 Q4_K_M) on the claim build's fast path, at FAST_THINKING_BUDGET 256 (arm Y) or 512 (arm Z), lift V-para answers vs the instruct 4B (C at L1, claim rows K at L2), and at what time cost?
From the record
- What was tested
- THINK4B
- Result
- +0.026 (−0.013…+0.068), ×3.7 time
- Verdict
- not detected: CI straddles zero below the test's minimum detectable effect: not evidence of no effect
- Detail
- closed: NO (near-lever), stopped after seed 42 (one seed, not the two-seed verdict the criterion specifies); CLOSED R10 relabels it 'not detected at MDE 0.06'
- Set
- V-para; v0.1 (smokes)
- Judge
- Claude Opus, Claude Haiku (L1)
- n
- 51 judged V-para ids (58 generated); slices 20
- Seeds
- s42 only (s43 requests prepared, not run)
- Pre-registered
- yes, sha256-frozen
- Reopen if
- BACKLOG s3: V-para s43 (Opus 102), only if the 4B side reopens
Linked facts · 3
- R10 Thinking 4B, V-para: Qwen3-4B-Thinking @512R10 Thinking 4B, V-para: Qwen3-4B-Thinking @512+0.026 (−0.013…+0.068), ×3.7 timeKinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatwhy closed: not detected at MDE 0.06Sourceresearch record
reports/CLOSED.md:25Open the data file - "20 hardest" slices inflated deltas: +0.091 slice vs +0.026 full (THINK4B); +0.089 vs +0.009 (THINK4B-V02)."20 hardest" slices inflated deltas: +0.091 slice vs +0.026 full (THINK4B); +0.089 vs +0.009 (THINK4B-V02).inflation ×3-10SetdevStatusdevJudgeOpus/Haikun20Seedsnot recordedDate2026-10-10Sourceresearch record
method/L1-PROTOCOL.md:28Open the data file - Thinking 4B (THINK4B, -V02): Qwen3-4B-Thinking at 256/512 → +0.026 / +0.009 (below +0.03; seed noise)Thinking 4B (THINK4B, -V02): Qwen3-4B-Thinking at 256/512 → +0.026 / +0.009 (below +0.03; seed noise)Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Sourcenot in the public repositories yetOpen the data file