Concept
held-out (blind) set
Exam questions that the people (and agents) building the app never see; the only fair test.
From the record
- In technical terms
- Item text stored outside builder-readable paths; run only by script, aggregates only; every touch logged in the exposure ledger.
Linked facts · 1
- Every touch of a held-out set was logged; both held-out sets are now development data.Every touch of a held-out set was logged; both held-out sets are now development data.v0.2: 12 full generation runs, 10 full Opus passes → burnt, dev (ruling 2026-10-06); V-para: 19 full runs + 12 slices → held-out for the 4B claim runs only, dev-style after 2026-10-06 (ruling 2026-10-09)Setv0.2; V-paraStatusburnt / dev-styleJudgenot recordedn110 / 58 (51 judged + 7 venue)Seedsnot recordedDate2026-10-09Sourceresearch record
reports/SET-STATUS.md:14research recordreports/SET-STATUS.md:15Open the data file