RatLabA Boar Research
Closed recordas of what’s still open →

Decision ·

Publish LabRat's numbers, keep the app maintainer's internal; headline = DEVICE-FINAL; b4ef94f3 desktop run = history; applyShown documented, not fixed

Numbers of BoarLite/LabRat (incl. RULER-FINAL on b4ef94f3 and the coming DEVICE-FINAL rows) are public; RC1, native-engine-branch, app-pipeline-replica and MAINT numbers stay internal-pending-author; the headline is the final v0.4 ruler on the phone; the desktop b4ef94f3 run is history; Citations.applyShown is a known limitation

From the record

Context
decided by: author (publication decisions 1-4, 2026-10-25)
Why
LabRat is the author's own app; the maintainer's numbers need his consent; a fix would invalidate the measured engine
What followed
and E-473 public; stays internal (carries the native branch's absolute)
Reversible
yes
Status
kept
Alternatives considered
  • keep all app numbers internal
  • publish everything
  • rerun the desktop ruler on LabRat code
  • fix applyShown and re-measure

Linked facts · 3

  • Author publication decisions of 2026-10-25: numbers of the author's own submission (BoarLite, renamed LabRat) are publishable, numbers of the app maintainer's app stay internal; the headline is the final v0.4 ruler measu…Author publication decisions of 2026-10-25: numbers of the author's own submission (BoarLite, renamed LabRat) are publishable, numbers of the app maintainer's app stay internal; the headline is the final v0.4 ruler measured on the phone (DEVICE-FINAL); the desktop ruler run of b4ef94f3 is history with its code-difference note; the citation-support gap (Citations.applyShown) is documented as a known limitation and not fixed before the measurementLabRat numbers public; headline = DEVICE-FINAL; applyShown not fixedDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho: author. Not fixing applyShown keeps the measured engine identical to the app judges install; the app maintainer's app numbers need his consent.Sourcenot in the public repositories yetOpen the data file
  • RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 …RULER-FINAL U-total of BoarLite b4ef94f3 on v0.4: Claude family 0.164 (0.123-0.208), OpenAI gpt-5.6-sol family 0.245 (0.194-0.300); minimum over families 0.164 (0.123-0.206); win-rate 0 of 71 (Claude) and 0 of 71 with 1 tie (gpt-5.6-sol); fixed-text refusals on 3 itemsClaude 0.164; gpt-5.6-sol 0.245; min 0.164; win-rate 0Setv0.4 (held-out ruler; exposed)Statusheld-out ruler (v0.4; exposed)JudgeClaude Opus + OpenAI gpt-5.6-sol (each vs its own references)n71 headline itemsSeeds1 (42)Date2026-10-24 20:34CaveatMetric: ruler v0.4 U-total (71 headline items, health weight 0.5); never on one axis with the judge.v1 ratio or outcome labels. Desktop twin, one seed. Published by the author's decision of 2026-10-25 (AD-15 (b)): BoarLite/LabRat is the author's own submission. History, not the headline: desktop twin of engine b4ef94f3, one seed, before the Rat persona line and the cleanup; not a measurement of LabRat 7788924a. The headline is the on-phone run DEVICE-FINAL (OF-8).Sourcenot in the public repositories yetOpen the data file
  • Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler…Open engine bug, as of 2026-10-25: the citation-support step (Citations.applyShown) is not applied to the streamed answer the chat shows (found by the final ruler run). Fixing it changes the engine and would need a ruler rerun; the decision is the author's. RULER-FINAL scored, and DEVICE-FINAL records, what the chat showsopen; fix needs a ruler rerunDate2026-10-25Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)CaveatNo fix is in the record as of this update.Sourcenot in the public repositories yetOpen the data file