RatLabA Boar Research
Closed recordas of what’s still open →

Concept

FACT-AVAIL (information-available split)

Labels saying, for each ruler question, whether the answer is actually written in the installed knowledge packs.

Chapter 12 · A second ruler (bench v0.4)

From the record

In technical terms
Pre-registered per item: AVAILABLE (≥ 0.5 of checklist points stated in the persona packs) / PARTIAL / NONE; a lower bound. On the 71 headline items: 19 / 25 / 27 (strict 15 / 23 / 33). Labels are quarantined until the result card; counts only.

Linked facts · 2

  • Information-available labels on v0.4Information-available labels on v0.4AVAILABLE 19 / PARTIAL 25 / NONE 27 (strict 15/23/33) of 71Setv0.4 (quarantined labels)Statusheld-out ruler (v0.4; exposed to two engines 2026-10-24)JudgeClaude Haiku + Claude Opusnnot recordedSeedsnot recordedDate2026-10-21Caveatcounts only; lists stay quarantinedSourcenot in the public repositories yetOpen the data file
  • FACT-AVAIL on v0.4 headline (71)FACT-AVAIL on v0.4 headline (71)AVAILABLE 19 / PARTIAL 25 / NONE 27 (strict 15/23/33); humanities 13/16 NONESetv0.4 held-out (counts only)Statusfrozen ruler set (held-out for the ruler; counts only)JudgeClaude Haiku-low + Claude Opus labelsnnot recordedSeedsnot recordedDate10-21 21:18Caveatlower bound; labels quarantinedSourcenot in the public repositories yetOpen the data file