RatLabA Boar Research
Closed recordas of what’s still open →

The research · Act IV, From the bar to a product · Chapter 11

A smaller engine for 8 GB phones

On a real 8 GB phone the 4B was too slow and once killed by the operating system, so on 2026-10-20 the author made Qwen3-1.7B the main 8 GB engine, for latency and memory and explicitly not for quality.

Full thesis

Two days of screens found no small model, prompt, decoder or gate that closes the 1.7B's reading gap at the same phone time, and the product rule became "refuse by default when no source is kept, with a warning line on what is shown".

2026-10-18 → 10-21← In the article: §4 The cemetery of ideas

The ordinary phone said no

On a common 8 GB phone the model we had used all along needed half a minute or more per answer, and once the phone simply shut it down for lack of memory. A smaller model, with 1.7 billion parameters, answered in a quarter of a minute or less. Pixel 8a speed, 1.7B vs 4B1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s; 4B LMK kill at 4.4 GB RSSSetpublic dev questionsStatusdevJudgetimingnnot recordedSeedsnot recordedDate10-20Caveat4B 73/73 only after a rebootVERIFIER NOTE: Pre-BATCHRET, 9 public dev questions; not comparable with the 73-question p50 of 10-21.Sourcenot in the public repositories yetOpen the data file

Faster, and it reads worse

On 20 October the author made the small model the main engine for 8 GB phones. Not because it is better: on 51 practice questions the bigger model stays clearly ahead. Because a phone that answers in seconds, without crashing, is a phone people can use. Qwen3-1.7B = main 8 GB engine; 4B = optional "more complete" tieradopted; confirmed at 19:07 grillDate2026-10-20 ~11:50Kinda record, not a measurement (set, status, judge, n, seeds: not recorded)Caveatwho decided: author; why: Pixel 8a: 1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s, one LMK kill; alternatives: keep 4B default; Qwen2.5-1.5B (faster but weaker); reversible: yesSourcenot in the public repositories yetOpen the data file

Nothing closed the gap

Over two days dozens of tricks were tried on the small model: other small models, new prompts, different ways of picking words, confidence checks. None made it read as well as the bigger one at the same speed on the phone.

No source, no answer (by default)

The product rule changed too: when the phone's libraries hold nothing on the question, the app says so instead of guessing, and a button shows the model's unsourced guess under a warning. We counted what that button reveals: more wrong answers than right ones, so it sits behind a second tap and never appears for health questions.

AI-reviewed, not clinically reviewed. Nothing here is medical advice.

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

A smaller main engine for 8 GB phones

Qwen3-1.7B on 8 GB: chosen for latency and memory, not quality

On a Pixel 8a (8 GB) running our measurement app, the 4B decoded at 3.6–6 tok/s, answered in 29–46 s and was killed once by the low-memory killer at 4.4 GB RSS; Qwen3-1.7B decoded at 13–16 tok/s and answered in 13–16 s (9 public dev questions, before batched lookups). On 2026-10-20 the author made the 1.7B the main 8 GB engine and the 4B an optional tier. Batching the pack lookups (937 → 48 SQL calls per retrieval, byte-identical retrieval on 390/390) cut the 1.7B's time to first token on the phone from 9.12 to 5.55 s. On the 73-question phone run the 1.7B's p50 is 4.2 s with PSS 3.39 GB, but that p50 covers deterministic answers and declines (35 % of v0.1); its model-path p50 is about 12 s. The cost is reading: under Opus on v0.1 51 dev the 4B leads by +0.109 (+0.063…+0.160) on the judge.v1 floor-adjusted ratio, and at identical pack contexts it reads 0.650 vs 0.529 (Haiku raw).

Closed record · 1.0

Final · as of 2026-10-25 16:46 · 0 of 71 wins on the phone, U-total 0.165 (Claude) / 0.242 (gpt-5.6-sol), below the 0.50 bar

6.8 Latency and memory over quality: the 1.7B as main engine

A V-para diagnosis decomposed the 4B's 0.586 ratio-point gap into wrong-from-memory 0.309 (23 items), absent knowledge 0.239 (24), thin 0.038 (4) and refusals 0 (Opus, seed 42, labels ± 0.05), with the fact present in the installed kit for 6 of 51 items; the estimated ceiling of all 8 GB levers on V-para was ≈ +0.01–0.02 raw V-para 4B gap decompositiongap 0.586 ratio points: wrong 0.309 (23 items), absent 0.239 (24), thin 0.038 (4), refusals 0SetV-para (dev-style since 10-09)Statusdev-style since 2026-10-09JudgeClaude Opus judge.v1; LOSSMAP labelsn51Seedsnot recordedDate10-19Caveatsingle-seed labels, shares ±0.05Sourcenot in the public repositories yetOpen the data file. On a Pixel 8a the 4B answered in 29–46 s and was killed once at 4.4 GB RSS, the 1.7B in 13–16 s (9 public dev questions) Pixel 8a speed, 1.7B vs 4B1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s; 4B LMK kill at 4.4 GB RSSSetpublic dev questionsStatusdevJudgetimingnnot recordedSeedsnot recordedDate10-20Caveat4B 73/73 only after a rebootVERIFIER NOTE: Pre-BATCHRET, 9 public dev questions; not comparable with the 73-question p50 of 10-21.Sourcenot in the public repositories yetOpen the data file; the author made the 1.7B the main 8 GB engine on 2026-10-20. Under Opus on CV5 51 (judge.v1 floor-adjusted ratio of sums, one seed per rung) the 4B leads the 1.7B by +0.109 (+0.063…+0.160) and Gemma 4 E2B by +0.098 (+0.054…+0.146); the 1.7B's lead over a closed-book 1.5B is +0.090 (+0.009…+0.170) under Opus and +0.038 (−0.023…+0.098) under Haiku, i.e. judge-dependent Tier ladder under Opus4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170)Setv0.1 (dev)StatusdevJudgeClaude Opus judge.v1nnot recordedSeeds1Date2026-10-21CaveatHaiku: 1.7B−1.5B +0.038 (−0.023..+0.098), so judge-dependentSourcenot in the public repositories yetOpen the data file. Two-stage screens (GATE v2; the bar = the largest measured null-arm net + 2, after null arms "fixed" 23–31 % of problem items) Null arms already "fix" a quarter of screen problems23-31 %; bar = max null net + 2SetdevStatusdevJudgeClaude Haikunnot recordedSeedsnot recordedDate2026-10-20Caveatcalibration rests on 2 nullsSourcenot in the public repositories yetOpen the data file closed prompts, decoding, token-confidence gates (AUC 0.530), decoder grounding, fine-tuned readers and small-model swaps; Gemma 4 E2B's stage-2 verdict flips with the judge (Haiku −0.002 below the bar, Opus +0.001 above) Gemma 4 E2B vs Qwen3-1.7B on MAIN 97Haiku +0.028 (+0.006..+0.052) FAIL by 0.002; Opus +0.031 (+0.005..+0.058) GO by 0.001; pooled Opus +0.036 (+0.011..+0.062)SetMAIN 97 (dev)StatusdevJudgeClaude Haiku; Claude Opusnnot recordedSeeds2Date2026-10-21Caveatgain mostly fewer refusals; 2.8x phone timeSourcenot in the public repositories yetOpen the data file at 2.8× phone time. A groundedness gate cut visible confident errors 7 → 3 of 177 under an Opus re-judge (Haiku 10 → 4), false refusals 0/14, at ≈ +9.3 s per answer projected on the phone REFUSE-UNSUPPORTEDvisible confident errors 7 -> 3 of 177 under Claude Opus (re-judge of the 38 deciding answers; Claude Haiku 10 -> 4); false refusals 0/15; U on answered +0.030 (+0.005..+0.057, Haiku); coverage 0.435 -> 0.328; +9.3 s p50 projected on the Pixel 8aSetMAIN 97 + 80 self-tests, devStatusdevJudgeClaude Haiku 5.5-low; Claude Opus re-judge for the confident-error countn177Seedsnot recordedDate10-20CaveatGATELAT: no cheaper variant keeps AUCVerification note: Judge-dependence: Haiku 10 -> 4 of 177; Opus re-judge of the 38 deciding answers 7 -> 3 of 177; the claim text should quote 7 -> 3 with Opus named. +9.3 s is projected.Sourcenot in the public repositories yetOpen the data file. The reveal-anyway control returned 11 correct to 16 confident errors in an Opus census Show-anyway reveals under OpusMAIN census 11 correct: 16 CE (0.69; non-health 0.56, CI 0.22-1.35)SetMAIN 97 (dev)StatusdevJudgeClaude Opusnnot recordedSeedsnot recordedDate2026-10-21CaveatHaiku 7:19 on same rows AI-reviewed, not clinically reviewed.Sourcenot in the public repositories yetOpen the data file. No drift was detected over five turns DRIFT17: Does the 1.7B degrade over a session?CE-unwarned t1 0/10 vs t4-5 3/20, CI covers 0; one unsafe health error in both armsSetlongsession-v1 80 (dev)StatusdevJudgeClaude Haikunnot recordedSeedsnot recorded95 % CIrise CI 0.00..0.30Date2026-10-21Caveatcriterion frozen: yes (5ea4b345); arm verdict: KILLED (no drift detected) AI-reviewed, not clinically reviewed.Sourcenot in the public repositories yetOpen the data file.

Badge rows the 10 public fact rows this depth cites; a number here without a chip comes from a row that is not public yet (see /data/)
Claim and valueSetStatusJudgenSeeds95 % CIDate
V-para 4B gap decompositiongap 0.586 ratio points: wrong 0.309 (23 items), absent 0.239 (24), thin 0.038 (4), refusals 0V-para (dev-style since 10-09)dev-style since 2026-10-09Claude Opus judge.v1; LOSSMAP labels51not recorded10-19
Pixel 8a speed, 1.7B vs 4B1.7B 13-16 tok/s, answers 13-16 s; 4B 3.6-6 tok/s, 29-46 s; 4B LMK kill at 4.4 GB RSSpublic dev questionsdevtimingnot recordednot recorded10-20
Tier ladder under Opus4B−1.7B +0.109 (+0.063..+0.160); 4B−Gemma +0.098 (+0.054..+0.146); 1.7B−closed-book 1.5B +0.090 (+0.009..+0.170)v0.1 (dev)devClaude Opus judge.v1not recorded12026-10-21
Null arms already "fix" a quarter of screen problems23-31 %; bar = max null net + 2devdevClaude Haikunot recordednot recorded2026-10-20
Gemma 4 E2B vs Qwen3-1.7B on MAIN 97Haiku +0.028 (+0.006..+0.052) FAIL by 0.002; Opus +0.031 (+0.005..+0.058) GO by 0.001; pooled Opus +0.036 (+0.011..+0.062)MAIN 97 (dev)devClaude Haiku; Claude Opusnot recorded22026-10-21
REFUSE-UNSUPPORTEDvisible confident errors 7 -> 3 of 177 under Claude Opus (re-judge of the 38 deciding answers; Claude Haiku 10 -> 4); false refusals 0/15; U on answered +0.030 (+0.005..+0.057, Haiku); coverage 0.435 -> 0.328; +9.3 s p50 projected on the Pixel 8aMAIN 97 + 80 self-tests, devdevClaude Haiku 5.5-low; Claude Opus re-judge for the confident-error count177not recorded10-20
Show-anyway reveals under OpusMAIN census 11 correct: 16 CE (0.69; non-health 0.56, CI 0.22-1.35)MAIN 97 (dev)devClaude Opusnot recordednot recorded2026-10-21
DRIFT17: Does the 1.7B degrade over a session?CE-unwarned t1 0/10 vs t4-5 3/20, CI covers 0; one unsafe health error in both armslongsession-v1 80 (dev)devClaude Haikunot recordednot recordedrise CI 0.00..0.302026-10-21
Facts for V-para are mostly nowherein installed kit 6/51; all packs on disk 10/51; full English Wikipedia 9/34 absent itemsV-paradev-style since 2026-10-09coverage counts51not recorded10-19
Estimated ceiling of all 8 GB levers on V-paraabout +0.01-0.02 rawV-paradev-style since 2026-10-09estimatenot recordednot recorded10-19

In one sentence

For an 8 GB phone, speed and memory picked the small model; it reads worse, and we say so.

Turning point

2026-10-20 ~11:50, the author: "Qwen3-1.7B becomes the main engine".

On the calendar

46 dated events · 18 Oct → 22 Oct

  1. 19 Oct · 15:19Independent direction review: stop opening arms, consolidate, hold saferoute
  2. 19 Oct · 22:20V-para diagnosis: loss split wrong 0.309 / absent 0.239 / thin 0.038; little addressable on 8 GB
  3. 20 Oct · 11:50Pivot: Qwen3-1.7B becomes the main 8 GB engine after Pixel 8a speed runs; LITE-MAIN program opened
  4. 21 Oct · 03:17RESEARCH-PAGE-OUTLINE written (coverage × quality, 20 NEEDS markers)
  5. 21 Oct · 15:18RESEARCH-PAGE-DRAFTS: five page-ready sections
All 46 events
  1. 2026-10-18 00:16Publication content base built (13 chapters, data files) as v0.1 preprint
  2. 2026-10-18 00:41Author decisions on publication (byline, inspiration credit, cross-family judge name, no spend disclosure) applied
  3. 2026-10-18 00:41HONESTPROMPT set-A interim shows refusal collapse; traced to a completion-harness artefact (no chat template) and corrected 00:53
  4. 2026-10-18 00:53Literature pass on prompting 1-8B models for honesty without over-refusal (PROMPT-LIT), Haiku sweep + Opus verification
  5. 2026-10-18 00:55Publication reframed as living preprint v0.1 "as of 2026-10-18" (internal build state)
  6. 2026-10-18 01:03HONESTPROMPT set-A audit: honest Detailed prompt cuts wrong claims 0.231 -> 0.062
  7. 2026-10-18 07:36HONESTPROMPT closed; capability sentence does not make Detailed write code (route code asks instead)
  8. 2026-10-18 10:02CHATFAIL S1: routing fix for code/format asks PARTIAL on 4B (+0.122 detector, ~+0.10 audited), NO on 1.5B
  9. 2026-10-18 10:30D4 prompt (D1 + "always do what the user asks" capability rule) = D1 on v0.1 (+0.003); D4 becomes recommended Detailed prompt
  10. 2026-10-19 14:12SAFEROUTE fix raises gold-page reach 1 -> 21/30 but adds unsafe verbatim extracts; frozen rule FAIL kept on record, author releases the patch anyway (later held)
  11. 2026-10-19 15:08SMALLCLS: 1.7B hazard classifier; review later shows it names the right hazard on only 14/45
  12. 2026-10-19 15:19Independent direction review: stop opening arms, consolidate, hold saferoute
  13. 2026-10-19 19:00Engine PR updated with 8 commits (saferoute held), 1,586/1,586 tests
  14. 2026-10-19 22:20V-para diagnosis: loss split wrong 0.309 / absent 0.239 / thin 0.038; little addressable on 8 GB
  15. 2026-10-20 00:01CHAINWHEN NO: 4B source-pick before the answer +0.019
  16. 2026-10-20 00:12V-exact acceptance harness baseline 0.475 (Haiku, 2 seeds), venues 6/6
  17. 2026-10-20 11:50Pivot: Qwen3-1.7B becomes the main 8 GB engine after Pixel 8a speed runs; LITE-MAIN program opened
  18. 2026-10-20 12:47LITEDECLINE PARTIAL: engine's compact-tier declines; releasing all five raises U0 +0.170 but doubles confident errors (17 -> 34)
  19. 2026-10-20 16:14CMP30 / CMP30OFF on the Pixel 8a: 1.7B vs Qwen2.5-1.5B; with declines released 1.7B leads (correctness +0.46)
  20. 2026-10-20 17:54BATCHRET on Pixel 8a: 1.7B TTFT 9.12→5.55 s
  21. 2026-10-20 19:39Five 1.7B levers NO-GO at screen: honest prompt, short context, quote-first, greedy, vendor sampling; token confidence (AUC 0.53) and first-token entropy (0.49) do not flag errors
  22. 2026-10-20 19:47CE taxonomy: 20 of 27 1.7B confident errors had no relevant source
  23. 2026-10-20 21:03Model-usage audit: 33 findings, 6 P0 (stale CE metric, refusals counted as CE, unscreened candidate model)
  24. 2026-10-20 21:06REREAD: in refuse-by-default mode each old screen held only 4 visible confident errors; the 6-fix bar was unreachable; quote-first is a regression
  25. 2026-10-20 21:17Groundedness add-on gate (GATE17-SUPPORT, AUC 0.965) and REFUSE-UNSUPPORTED PASS: visible confident errors 7 -> 3 of 177 under the Claude Opus judge (10 -> 4 under Claude Haiku); +9.3 s p50 projected for the Pixel 8a (5.1 s measured on a desktop)
  26. 2026-10-20 22:01Judge v2 (binary key-point checklist) NO-GO by frozen test-retest rule
  27. 2026-10-20 22:09REFUSE-UNSUPPORTED PASS: groundedness gate cuts visible CEs 10→4 of 177
  28. 2026-10-20 22:29GATE v2: screens recalibrated against measured null runs (null alone fixes 23-31 %)
  29. 2026-10-20 23:46Gemma 4 E2B: GO on all four screens, stage 2 FAIL by 0.002
  30. 2026-10-21 00:00Overnight model screens: Q8 quant, Qwen3.5-2B, OCC-RAG, FOTAPOL LoRA, Jan-v1-edge, SmolLM3, granite-1B all NO; Ministral-3-3B + citation prompt stage-2 PASS at ~3x phone time
  31. 2026-10-21 02:20MIN3CITE stage 2 PASS (Ministral-3-3B + citation prompt), later shelved as dominated by the 4B
  32. 2026-10-21 02:23Gemma 4 E2B on the Pixel 8a: 73/73, answer p50 11.8 s; judged = desktop, 0.083 below the 4B on the same phone
  33. 2026-10-21 02:30Gemma on Pixel 8a = desktop Gemma, -0.083 vs the 4B on the same phone
  34. 2026-10-21 02:38Gemma 4 mobile deep research (10-agent workflow, 92 findings, 35 confirmed, 10 corrected): opt-in entry with thinking guards; 1.7B stays default
  35. 2026-10-21 02:44GEMMA-VS-4B on MAIN 97 (Haiku): Gemma captures 38 % of the 4B gain
  36. 2026-10-21 03:06Direction proposal reviewed by a 9-agent adversarial workflow: 71 findings stand, 5 fall, no wrong number
  37. 2026-10-21 03:17RESEARCH-PAGE-OUTLINE written (coverage × quality, 20 NEEDS markers)
  38. 2026-10-21 03:31Pixel 8a: Qwen3-1.7B on 73 questions p50 4.2 s vs Gemma 11.8 s
  39. 2026-10-21 03:38HON-1/HON-2: warning line precision 0.39; "show anyway" button gives ~7 correct vs 35 CE per 100 reveals (liability as specified)
  40. 2026-10-21 05:55GEMMA-OPUS-MAIN: Gemma − 1.7B +0.031 s42 under Opus, GO by 0.001
  41. 2026-10-21 07:24Morning report with 11 author decisions published (PT-BR)
  42. 2026-10-21 13:12GOAL-2026-10-21 set: raise U_8GB +0.03 (Opus) with guards; cut T_8GB 7.6 → 5.0 s
  43. 2026-10-21 13:181.7B engine lines closed one by one at nulls (X2b, routing, keep, coverage reasons, first-aid cards, keep-E)
  44. 2026-10-21 15:18RESEARCH-PAGE-DRAFTS: five page-ready sections
  45. 2026-10-21 22:10DECODE-GROUND closed: decoder-level grounding on the 1.7B at the null
  46. 2026-10-22 16:19Device day report: only the 4B speed row ran (76 s p50, 4.77 tok/s)

Related nodes on the map

This chapter on the map →

Experiments

Decisions

.md