Qwen3.8-Flash-Next UD-Q4_K_XL on H200
GGUF Q8_0+Q4_K
Engines
llama.cpp
b11571 · 097cf33d9 ↗measured 2026-11-09 23:06 UTC
100.4 tok/s
14% of the roofline709.1 tok/s possible
- now, a token
- 9.96 ms
- floor
- 1.41 ms
- read, a token
- 6.57 GB
Speed at context
Where the time goes
- Now, a token
- 9.96 ms
- Roofline floor, a token
- 1.41 ms
- Gap to the floor
- 8.55 ms
- Kernels a token
- 498
Launch heavy: 3472 kernel launches per token
Cannot split: ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, attn_qo BF16, attn_kv BF16, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, ffn_down Q8_0, moe_shared_expert_down Q8_0, attn_qo BF16, attn_kv BF16, moe_router F32. No kernel group matched their calls and bytes
Not classified: blk.*.hc_attn_down.weight Q8_0, blk.*.hc_attn_up.weight Q8_0, blk.*.ple_key.weight Q8_0, blk.*.ple_value.weight Q8_0, output_hc_down.weight Q8_0, output_hc_up.weight Q8_0. 100 tensors, 0.38 GB per token, not classified; counted in the limit
Per role ms a token. floor (bytes ÷ measured memory speed) above the floor
Per kernel (3): ms now / ms at the roofline, launches a token
No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.
Run h200-flash-next-q4kxl-llama
Tested on
- GPU
- H200 · 140 GB
- System RAM
- 1511 GB
- CPU threads
- 256
- Memory, measured
- 4662 GB/s
- Temp, max
- see notes
- Power
- see notes
- SM clock
- –
- Driver
- 595.91.07
- Measured
- 2026-11-09 23:06 UTC · context 512 tokens · measured, in-model (nsys): 12 of 21 weight roles timed
Qwen3.8-Flash-Next on other GPUs
The fastest setup measured on each GPU, one user. The percentage is that GPU's own roofline.