boltbeam.fyi

Qwen3.8-27B UD-Q4_K_M on H200 NVL

GGUF IQ4_XS+Q5_K

The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 21:00 UTC

78.6 tok/s

27% of the roofline286.7 tok/s possible

now, a token
12.72 ms
floor
3.49 ms
read, a token
15.74 GB
More
memory, measured
4513 GB/s

What was run

llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp79.5 tok/s of 287.2
512 tokens
llama.cpp78.6 tok/s of 286.7
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
12.72 ms
Roofline floor, a token
3.49 ms
Gap to the floor
9.23 ms
Kernels a token
497

Launch heavy: 1914 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up IQ4_XS1.84 / 0.64 ms 35%
ffn_gate_up Q4_K0.98 / 0.31 ms 32%
ffn_gate_up Q5_K1.05 / 0.38 ms 36%
ffn_down Q5_K0.77 / 0.30 ms 39%
attn_qkv Q4_K0.63 / 0.19 ms 30%
ffn_down IQ4_XS0.63 / 0.25 ms 40%
lm_head Q6_K0.58 / 0.23 ms 40%
ssm_alpha_beta Q8_00.32 / 0.01 ms 2%
ffn_gate_up Q3_K0.34 / 0.05 ms 15%
ssm_out Q5_K0.44 / 0.16 ms 36%
attn_gate Q4_K0.33 / 0.09 ms 29%
ffn_down Q4_K0.33 / 0.11 ms 34%
attention0.19 / 0.01 ms 4%
attn_qkv Q5_K0.27 / 0.10 ms 36%
attn_gate Q5_K0.17 / 0.06 ms 34%
attn_qo Q4_K0.15 / 0.05 ms 31%
attn_gate IQ4_XS0.15 / 0.04 ms 30%
ffn_down IQ3_S0.12 / 0.03 ms 22%
attn_qkv IQ4_XS0.12 / 0.04 ms 31%
attn_qo Q6_K0.09 / 0.02 ms 25%
ffn_gate_up IQ4_NL0.09 / 0.03 ms 36%
ssm_out Q6_K0.09 / 0.03 ms 33%
ffn_down IQ4_NL0.09 / 0.03 ms 39%
ffn_down Q3_K0.06 / 0.01 ms 14%
attn_kv Q6_K0.06 / 0.01 ms 20%
ssm_out Q4_K0.06 / 0.02 ms 31%
ssm_out IQ4_XS0.06 / 0.02 ms 32%
attn_kv Q5_K0.05 / 0.01 ms 19%
ffn_gate_up IQ3_S0.04 / 0.01 ms 20%
ffn_down Q6_K0.05 / 0.02 ms 32%
ffn_gate_up Q6_K0.04 / 0.02 ms 37%
attn_kv Q8_00.03 / 0.01 ms 29%
attn_qkv IQ4_NL0.02 / 0.01 ms 34%
attn_kv Q4_K0.00 / 0.00 ms 17%
quantize0.65 ms · no floor
norm0.48 ms · no floor
copy0.40 ms · no floor
attn qo0.26 ms · no floor
recurrent0.22 ms · no floor
7 other small kernels0.48 ms · no floor
Per kernel (8): ms now / ms at the roofline, launches a token
mul_mat_vec_q · IQ4_XS3.47 ms · no floor ×117
mul_mat_vec_q · Q5_K3.40 ms · no floor ×131
mul_mat_vec_q · Q4_K2.87 / 0.77 ms ×103
mul_mat_vec_q · Q6_K0.99 / 0.33 ms ×24
mul_mat_vec_q · Q8_00.93 / 0.02 ms ×104
mul_mat_vec_q · Q3_K0.36 / 0.06 ms ×7
mul_mat_vec_q · IQ4_NL0.22 / 0.07 ms ×7
mul_mat_vec_q · IQ3_S0.17 / 0.03 ms ×4

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h200nvl-in-model

Tested on

GPU
H200 NVL · 140 GB
System RAM
1008 GB
CPU threads
224
Memory, measured
4513 GB/s
Temp, max
59 °C
Power
126 W mean of 500 W
SM clock
1780 MHz
Driver
615.78.08
Measured
2026-11-09 21:00 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed

Qwen3.8-27B on other GPUs