boltbeam.fyi

Qwen3.8-Flash-Next GSQ-RCO IQ1_M on RTX 5090

GGUF GSQ-RCO IQ1_M, 2 shards, 58.4 GB

Engines

llama.cppv1.0.0 (b1b770d9)

77.2 tok/s

14% of the roofline537.1 tok/s possible

now, a step
44.53 ms
floor
6.40 ms
read, a step
5.76 GB
More
memory, measured
1707 GB/s
tokens a step
3.44 (MTP draft)
prefill per turn
2.61 s
time to first token
2.71 s
whole session
485 s
stock: -c 131072, q8_0 KV, MTP draft head, every other setting default

llama.cpp + modsv1.0.0 + 13 mods · tuned by BoltBeam 0.3.26

226.1 tok/s

43% of the roofline528.5 tok/s possible

now, a step
14.96 ms
floor
6.40 ms
read, a step
5.76 GB
More
memory, measured
1707 GB/s
tokens a step
3.38 (MTP draft)
prefill per turn
0.83 s
time to first token
0.93 s
whole session
166 s
-c 131072 -ctk/-ctv q8_0 -fa on -ngl 99 -ncmoe 16 -ub 1024 -b 2048 --moe-cache-mib 6240 -lm none, MTP draft 3 tokens/step, 13 mods on. Settings derived by boltbeam tune for this machine.

Speed at context

<32k
llama.cpp96.0 tok/s of 544.2
llama.cpp + mods235.8 tok/s of 539.8
32-64k
llama.cpp86.0 tok/s of 551.7
llama.cpp + mods234.4 tok/s of 542.7
64-96k
llama.cpp74.2 tok/s of 545.7
llama.cpp + mods229.0 tok/s of 535.1
96-128k
llama.cpp61.3 tok/s of 510.2
llama.cpp + mods207.9 tok/s of 499.8
Decode tok/s by context depth at the start of each turn, over a 59-turn session that grows to 129k tokens. The pink tick is the roofline.

Where the time goes

Now, a step
14.96 ms
Roofline floor, a step
6.40 ms
Gap to the floor
8.56 ms
Kernels a step
3648

One third of the gap to the floor is not kernels: ~1.8 ms a step of host idle, plus expert-cache misses.

Per role ms a step. floor (bytes ÷ measured memory speed) above the floor

dense weights, verify4.07 / 2.09 ms 51%
expert weights, resident + cached2.58 / 1.90 ms 74%
MTP draft passes1.16 / 0.34 ms 29%
draft head0.93 / 0.92 ms 99%
verify attention + KV write0.82 / 0.06 ms 8%
verify head0.32 / 0.31 ms 97%
recurrent / GDN layers0.24 / 0.13 ms 54%
small and elementwise kernels2.20 ms · no floor
host gaps and launch overhead1.80 / 0.05 ms 3%
verify top-k + mask0.79 / 0.01 ms 2%
expert-cache misses, host to GPU0.56 ms · no floor
verify indexer0.25 / 0.03 ms 12%
rollback copies0.05 ms · no floor

No kernel capture for this run.

Run flash-next-128k-mods

Tested on

GPU
RTX 5090 · 32 GB
System RAM
31 GB
CPU threads
16
Memory, measured
1707 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.99.02
Measured
2026-11-17 · context 131072 tokens · 59-turn session to 129.3k tokens, 3 paired runs, temperature 0
  • GPU power capped at 450 W, reset before every run
  • start gate: GPU at or under 70 °C, board under 58 °C
  • CPU: Ryzen 7 5800X · RAM: 32 GB DDR4-2667 · PCIe 4.0 x16 (B550)
  • MoE model larger than VRAM: 16 layers of experts in system RAM
  • roofline: 5.76 GB a decode step (3.38 tokens) / 1,707 GB/s measured = 6.4 ms

Qwen3.8-Flash-Next on other GPUs

Qwen3.8-Flash-Next has only been measured on the RTX 5090 so far.