RatLabA Boar Research
Closed recordas of what’s still open →

Concept

expert streaming (bmoe, LES)

Loading only the expert blocks a word needs from storage, instead of keeping the whole big model in memory.

Chapter 7 · A phone on a desk, and in a hand

From the record

In technical terms
bmoe v0.28.0 (the on-device engine) is judged-equivalent to llama-server on identical contexts. An OpenBMB llama.cpp fork (LES) runs the 35B in ≈ 2.8 GiB RSS on a desktop CPU at 14–17 tok/s, judged-equivalent; phone flash speed unmeasured.

Linked facts · 2

  • The on-device MoE engine (bmoe) is not token-identical to the desktop server but is judged equivalent on identical contexts.The on-device MoE engine (bmoe) is not token-identical to the desktop server but is judged equivalent on identical contexts.0.623 vs 0.634, Δ −0.012 (−0.045…+0.020); greedy token-identical 0/10Setv0.1Statuspublic devJudgeClaude Opus (judge.v1, both orders)n57Seeds1Date2026-10-07CaveatAn equivalence, not a lever: lets desktop deep-tier numbers transfer to a bmoe device plan with this caveat.Sourceresearch record reports/CLOSED.md:35research record reports/CLOSED.md#R16Open the data file
  • An expert-streaming engine runs the 35B model in under 3 GiB of RAM on a desktop CPU with judged-equivalent answers.An expert-streaming engine runs the 35B model in under 3 GiB of RAM on a desktop CPU with judged-equivalent answers.judged 0.625 vs llama-server 0.634, paired −0.010 (−0.042…+0.018); RSS 2.75–2.81 GiB; 14–17 tok/s on the Mac mini CPU; 10/10 greedy-identical vs the fork baseSetv0.1Statuspublic devJudgeClaude Opus (judge.v1, both orders)n57 (R450 contexts)Seeds1Date2026-10-16CaveatDesktop only, experts read from a fast SSD and page cache; phone flash speed is unmeasured (analytic bound ≈ 4–6 tok/s with compute).Sourcenot in the public repositories yetOpen the data file