Experiment ·
XFAM
Does a judge from another model family change the levels, ranking or win-rate of the 4B and deep-tier rows on V-para?
From the record
- What was tested
- cross-family re-judge of the existing V-para traces
- Result
- 4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01
- Verdict
- measurement: measures a property; no lever verdict
- Set
- V-para (dev-style)
- Judge
- cross-family (OpenAI gpt-5.6-sol (via the Codex CLI))
- Pre-registered
- no (post-hoc)
Linked facts · 2
- A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate.A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate.4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01SetV-paraStatusdev-styleJudgecross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered)n51 (408 judge calls)Seeds2Date2026-10-16CaveatPost-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline.Sourcenot in the public repositories yetOpen the data file
- Cross-family judge via paid API (GPT pilot): 204-call pilot → held / withdrawn (no paid keys, no human review) (Later done by the author via Codex (XFAM))Cross-family judge via paid API (GPT pilot): 204-call pilot → held / withdrawn (no paid keys, no human review) (Later done by the author via Codex (XFAM))Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Sourcenot in the public repositories yetOpen the data file