boltbeam.fyi

Qwen3-14B Q4_K_M on H100 80GB HBM3

GGUF Q4_K+Q6_K

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 18:19 UTCnewest release then: 0.9.0 (checked 2026-11-09)

144.4 tok/s

39% of the roofline374.3 tok/s possible

now, a token
6.93 ms
floor
2.67 ms
read, a token
8.65 GB
More
memory, measured
3237 GB/s

What was run

llama-bench -m Qwen3-14B-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp144.7 tok/s of 377.0
512 tokens
llama.cpp144.4 tok/s of 374.3
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
6.93 ms
Roofline floor, a token
2.67 ms
Gap to the floor
4.25 ms
Kernels a token
281

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

Per kernel (2): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q4_K5.40 ms · no floor ×240
mul_mat_vec_q · Q6_K1.38 ms · no floor ×41

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run h100-qwen3-14b-q4km-llama

Tested on

GPU
H100 80GB HBM3 · 80 GB
System RAM
443 GB
CPU threads
52
Memory, measured
3237 GB/s
Temp, max
38 °C
Power
143 W mean of 700 W
SM clock
1966 MHz
Driver
580.126.20
Measured
2026-11-09 18:19 UTC · context 512 tokens · measured, in-model (nsys): 7 of 7 weight roles timed

Qwen3-14B on other GPUs