boltbeam.fyi

Qwen3-14B Q4_K_M on H200 NVL

GGUF Q4_K+Q6_K

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 21:16 UTCnewest release then: 0.9.0 (checked 2026-11-09)

135.0 tok/s

26% of the roofline521.8 tok/s possible

now, a token
7.41 ms
floor
1.92 ms
read, a token
8.65 GB
More
memory, measured
4513 GB/s

What was run

llama-bench -m Qwen3-14B-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp136.9 tok/s of 525.6
512 tokens
llama.cpp135.0 tok/s of 521.8
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
7.41 ms
Roofline floor, a token
1.92 ms
Gap to the floor
5.49 ms
Kernels a token
281

Launch heavy: 766 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up Q4_K2.92 / 0.89 ms 30%
ffn_down Q6_K0.99 / 0.32 ms 33%
attn_kv Q4_K0.60 / 0.04 ms 7%
attn_qo Q4_K0.82 / 0.26 ms 32%
ffn_down Q4_K0.65 / 0.22 ms 34%
lm_head Q6_K0.36 / 0.14 ms 40%
attention0.16 / 0.02 ms 12%
attn_kv Q6_K0.10 / 0.02 ms 19%
quantize0.39 ms · no floor
norm0.29 ms · no floor
KV cache write0.07 ms · no floor
kernel gaps, at most0.04 ms · no floor
copy0.00 ms · no floor
elementwise0.00 ms · no floor
Per kernel (2): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q4_K5.62 / 1.41 ms ×240
mul_mat_vec_q · Q6_K1.40 / 0.48 ms ×41

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h200-qwen3-14b-q4km-llama

Tested on

GPU
H200 NVL · 140 GB
System RAM
1008 GB
CPU threads
224
Memory, measured
4513 GB/s
Temp, max
55 °C
Power
121 W mean of 500 W
SM clock
1785 MHz
Driver
615.78.08
Measured
2026-11-09 21:16 UTC · context 512 tokens · measured, in-model (nsys): 7 of 7 weight roles timed

Qwen3-14B on other GPUs