Concept

judge family / self-preference

The questions, the reference answers and the judge all came from the same AI company, which may bias the scores.

From the record

In technical terms: Claude wrote v0.2 and V-para items and the references, and judged. Self-preference direction is measured only by one post-hoc cross-family re-judge. v0.3 is to be written outside the Claude family.

A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate.A judge from a different model family scores everything higher on V-para, but keeps the ranking and the near-zero win-rate.4B: raw 0.497 (0.449–0.547), floor-adjusted 0.361; deep + think 512: raw 0.667 (0.621–0.711), floor-adjusted 0.577 (0.519–0.632); deep − 4B +0.171 (+0.123…+0.219) vs +0.139 under Claude; per-item Spearman 0.88–0.92; win-rate 0.00–0.01SetV-paraStatusdev-styleJudgecross-family: OpenAI gpt-5.6-sol (via the Codex CLI) (post-hoc, not pre-registered)n51 (408 judge calls)Seeds2Date2026-10-16CaveatPost-hoc and not pre-registered; it is the only point above 0.50 floor-adjusted on V-para and exists only under the cross-family judge; the lower-judge rule keeps the Claude number as headline.Sourcenot in the public repositories yetOpen the data file