Experiment ·
DEEPPAR-L1
D4 step (c): at the judged level, does bmoe v0.28.0 produce answers of the same quality as llama-server b11010 (control A deep35 R450; B = 4B R450) on identical R450 gold-passage prompts?
From the record
- What was tested
- DEEPPAR-L1
- Result
- 0.623 vs 0.634, Δ −0.012 (−0.045…+0.020); greedy token-identical 0/10
- Verdict
- parity: an equivalence (same quality, cheaper or transferable), not a gain
- Detail
- closed — TRANSFER (judged parity; equivalence shown, not a lever)
- Set
- v0.1 EN 57 (R450 oracle contexts)
- Judge
- Claude Opus
- n
- 57
- Seeds
- 1 (seed 42, temp 0.7)
- Pre-registered
- yes, sha256-frozen
- Reopen if
- none: D2/D3/D5 wait for a device
Linked facts · 2
- The on-device MoE engine (bmoe) is not token-identical to the desktop server but is judged equivalent on identical contexts.The on-device MoE engine (bmoe) is not token-identical to the desktop server but is judged equivalent on identical contexts.0.623 vs 0.634, Δ −0.012 (−0.045…+0.020); greedy token-identical 0/10Setv0.1Statuspublic devJudgeClaude Opus (judge.v1, both orders)n57Seeds1Date2026-10-07CaveatAn equivalence, not a lever: lets desktop deep-tier numbers transfer to a bmoe device plan with this caveat.Sourceresearch record
reports/CLOSED.md:35research recordreports/CLOSED.md#R16Open the data file - R16 bmoe vs llama-server parity: judged on identical contextsR16 bmoe vs llama-server parity: judged on identical contexts−0.012 (−0.045…+0.020)Kinda record, not a measurement (set, status, judge, n, seeds, date: not recorded)Caveatwhy closed: equivalence shown (not a lever)Sourceresearch record
reports/CLOSED.md:35Open the data file