Experiment ·
COMPLETE
Does a completeness instruction (120-220 words, max tokens 700) on the deep35 reader with gold R450 contexts raise the judge ratio vs the identical DEEPREAD run (short-answer prompt) without losing correctness?
From the record
- What was tested
- COMPLETE
- Result
- −0.014 (−0.055…+0.024); confident-wrong 7 → 15
- Verdict
- closed: rule failed, CI excluded the hoped-for gain, a step-0 gate failed, or the mechanism was falsified
- Detail
- closed: NO by the frozen rule (ratio 0.620 < 0.700; correctness 3.10 vs >= 3.80; confident-wrong 7 → 15 vs allowed +2)
- Set
- v0.1 EN 57
- Judge
- Claude Opus
- n
- 57 (v0.1 EN)
- Seeds
- 1 (seed 42)
- Pre-registered
- yes, sha256-frozen
- Reopen if
- none stated; DEEPCOVER (closed NO 10-12, R67; key-point coverage, BACKLOG B2) was the only distinct variant
Linked facts · 2
- R8 Completeness instruction: deep reader + gold passagesR8 Completeness instruction: deep reader + gold passages−0.014 (−0.055…+0.024); confident-wrong 7 → 15Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatwhy closed: mechanism raises wrong claimsSourceresearch record
reports/CLOSED.md:23Open the data file - Completeness prompt on the 35B (COMPLETE): 120–220 words → correctness −0.79, CW 7 → 15 (prompting buys completeness with errors)Completeness prompt on the 35B (COMPLETE): 120–220 words → correctness −0.79, CW 7 → 15 (prompting buys completeness with errors)Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Sourceresearch record
reports/CLOSED.md:23Open the data file