Experiment ·
DEEPREAD
Does the deep model (Qwen3.6-35B-A3B UD-Q2_K_XL) convert gold retrieved passages (R450 oracle contexts) better than the 4B (Qwen3-4B-Instruct-2507) reading the identical contexts, on v0.1 EN 57?
From the record
- What was tested
- DEEPREAD
- Result
- +0.029 (−0.004…+0.063)
- Verdict
- not detected: CI straddles zero below the test's minimum detectable effect: not evidence of no effect
- Detail
- INCONCLUSIVE (frozen band 0.606 ≤ x < 0.660; deep35 R450 0.6345); CLOSED R15 records it as 'not detected at MDE 0.055'
- Set
- v0.1 EN 57
- Judge
- Claude Opus
- n
- 57
- Seeds
- 1 (seed 42)
- Pre-registered
- yes, sha256-frozen
- Reopen if
- none stated
Linked facts · 1
- R15 Deep reader advantage: gold passagesR15 Deep reader advantage: gold passages+0.029 (−0.004…+0.063)Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatwhy closed: not detected at MDE 0.055Sourceresearch record
reports/CLOSED.md:34Open the data file