Experiment ·
SFT4B
Can a small LoRA (rank 16, ~1,200 extraction-style grounded QA examples from the packs' own text, filtered by jev quote_supports >= 0.9) teach the claim 4B reader to use the kept pack passage, measured as X (claim 4B + LoRA) vs C (claim 4B, SELFTEST-SCORES with-pack traces) on the 180 pack-matched self-tests, with v0.1 public 60 no-harm.
From the record
- What was tested
- SFT4B
- Result
- pass 38 %, 562 kept vs ≥ 1,000
- Verdict
- closed: rule failed, CI excluded the hoped-for gain, a step-0 gate failed, or the mechanism was falsified
- Detail
- open / in progress: stage 0 data PASS (v1, then v2); stage 1 environment + parity tooling PASS on the smoke adapter (STAGE1.md); training (step 6) on hold until v3 data (addendum 2); no verdict
- Set
- 180 self-tests; v0.1
- Judge
- none (deterministic gate)
- n
- 180 (eval, planned); 1,200 train / 100 hold-out examples built
- Seeds
- 1 (deterministic generation; training seed 42)
- Pre-registered
- yes, sha256-frozen
- Reopen if
- the deep-tier teacher was probed 10-14 (120 items): 6/120 pass vs Haiku 54/120 → not a stronger teacher as run; remaining: an Opus teacher (≥ 50 % round-1 at quote_supports ≥ 0.9) or a 0.8 bar validated by the strict 20-item review ≤ 6/20
Linked facts · 3
- R71 Grounded-reading SFT with Haiku teacher: data gatesR71 Grounded-reading SFT with Haiku teacher: data gatespass 38 %, 562 kept vs ≥ 1,000Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatwhy closed: stage-0 FAILSourceresearch record
reports/CLOSED.md:100Open the data file - SFT4B: LoRA fine-tune of the 4B to read kept passages → teacher faithfulness 38 % (Haiku), 5 % (deep) vs 50 % (no faithful teacher)SFT4B: LoRA fine-tune of the 4B to read kept passages → teacher faithfulness 38 % (Haiku), 5 % (deep) vs 50 % (no faithful teacher)Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Sourcenot in the public repositories yetOpen the data file
- Reader-side fine-tuning teacher faithfulness (SFT4B)Reader-side fine-tuning teacher faithfulness (SFT4B)Haiku teacher 38–45 %; deep-tier teacher 6/120 (5 %) vs 50 % barSetdistractor-rich engine renderingsJudgejevDate2026-10-13 22:28 / 10-14 10:56Kinda record, not a measurement (status, n, seeds: not recorded)Caveat45/120 deep answers empty (shared token budget)Sourcenot in the public repositories yetOpen the data file