boltbeam.fyi

Qwen3.8-27B UD-Q4_K_M on H100 80GB HBM3

GGUF IQ4_XS+Q5_K

The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 18:22 UTCnewest release then: 0.9.0 (checked 2026-11-09)

83.6 tok/s

41% of the roofline205.7 tok/s possible

now, a token
11.96 ms
floor
4.86 ms
read, a token
15.74 GB
More
memory, measured
3237 GB/s

What was run

llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp83.4 tok/s of 206.0
512 tokens
llama.cpp83.6 tok/s of 205.7
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
11.96 ms
Roofline floor, a token
4.86 ms
Gap to the floor
7.10 ms
Kernels a token
497

Launch heavy: 1914 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up IQ4_XS1.71 / 0.89 ms 52%
ffn_gate_up Q4_K0.91 / 0.43 ms 48%
ffn_gate_up Q5_K0.98 / 0.53 ms 54%
ffn_down Q5_K0.74 / 0.42 ms 56%
attn_qkv Q4_K0.59 / 0.26 ms 45%
ssm_alpha_beta Q8_00.30 / 0.01 ms 3%
ssm_out Q5_K0.47 / 0.22 ms 47%
ffn_gate_up Q3_K0.31 / 0.07 ms 23%
ffn_down IQ4_XS0.59 / 0.35 ms 60%
lm_head Q6_K0.54 / 0.32 ms 60%
attention0.18 / 0.01 ms 6%
attn_gate Q4_K0.30 / 0.13 ms 43%
ffn_down Q4_K0.32 / 0.15 ms 49%
attn_qkv Q5_K0.25 / 0.13 ms 54%
attn_gate Q5_K0.16 / 0.08 ms 51%
attn_qo Q4_K0.14 / 0.07 ms 47%
attn_gate IQ4_XS0.14 / 0.06 ms 46%
ffn_down IQ3_S0.11 / 0.04 ms 33%
attn_qkv IQ4_XS0.11 / 0.05 ms 47%
attn_qo Q6_K0.09 / 0.03 ms 36%
ssm_out Q6_K0.09 / 0.04 ms 46%
ffn_down Q3_K0.06 / 0.01 ms 21%
ffn_gate_up IQ4_NL0.09 / 0.05 ms 54%
attn_kv Q6_K0.05 / 0.02 ms 30%
ssm_out Q4_K0.06 / 0.03 ms 42%
ffn_down IQ4_NL0.08 / 0.05 ms 57%
attn_kv Q5_K0.04 / 0.01 ms 28%
ssm_out IQ4_XS0.06 / 0.03 ms 45%
ffn_gate_up IQ3_S0.04 / 0.01 ms 30%
ffn_down Q6_K0.05 / 0.02 ms 46%
attn_kv Q8_00.03 / 0.01 ms 41%
ffn_gate_up Q6_K0.04 / 0.02 ms 56%
attn_qkv IQ4_NL0.02 / 0.01 ms 53%
attn_kv Q4_K0.00 / 0.00 ms 24%
quantize0.63 ms · no floor
norm0.46 ms · no floor
copy0.37 ms · no floor
attn qo0.24 ms · no floor
recurrent0.21 ms · no floor
7 other small kernels0.41 ms · no floor
Per kernel (8): ms now / ms at the roofline, launches a token
mul_mat_vec_q · IQ4_XS3.24 ms · no floor ×117
mul_mat_vec_q · Q5_K3.23 ms · no floor ×131
mul_mat_vec_q · Q4_K2.70 / 1.08 ms ×103
mul_mat_vec_q · Q6_K0.93 / 0.46 ms ×24
mul_mat_vec_q · Q8_00.90 / 0.02 ms ×104
mul_mat_vec_q · Q3_K0.32 / 0.08 ms ×7
mul_mat_vec_q · IQ4_NL0.20 / 0.10 ms ×7
mul_mat_vec_q · IQ3_S0.16 / 0.05 ms ×4

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h100-qwen3.8-27b-q4km-llama

Tested on

GPU
H100 80GB HBM3 · 80 GB
System RAM
443 GB
CPU threads
52
Memory, measured
3237 GB/s
Temp, max
40 °C
Power
147 W mean of 700 W
SM clock
1980 MHz
Driver
580.126.20
Measured
2026-11-09 18:22 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed

Qwen3.8-27B on other GPUs