Experiment ·
XFAM-V02
Does the cross-family offset hold on v0.2?
From the record
- What was tested
- cross-family re-judge on v0.2
- Result
- 4B: raw 0.617 (0.567–0.666), floor-adjusted 0.517 (2 seeds); deep + think 512: raw 0.714 (0.666–0.759), floor-adjusted 0.639 (0.579–0.695) (1 seed); deep − 4B +0.097 vs +0.096 under Claude; Spearman 0.90–0.97; win-rate 2.7 % / 6.4 % vs 0 % under Claude
- Verdict
- measurement: measures a property; no lever verdict
- Set
- v0.2 (dev)
- Judge
- cross-family (OpenAI gpt-5.6-sol (via the Codex CLI))
- Pre-registered
- no (post-hoc)
Linked facts · 1
- The cross-family judge on v0.2.The cross-family judge on v0.2.4B: raw 0.617 (0.567–0.666), floor-adjusted 0.517 (2 seeds); deep + think 512: raw 0.714 (0.666–0.759), floor-adjusted 0.639 (0.579–0.695) (1 seed); deep − 4B +0.097 vs +0.096 under Claude; Spearman 0.90–0.97; win-rate 2.7 % / 6.4 % vs 0 % under ClaudeSetv0.2Statusdev (burnt)Judgecross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered)n110 (660 judge calls)Seeds4B 2, deep 1Date2026-10-17CaveatPost-hoc. The Claude judge is the stricter family by ≈ 0.10 on both sets; the offset is stable across sets and arms.Sourcenot in the public repositories yetOpen the data file