boltbeam.fyi

Qwen3-14B Q4_K_M on RTX 3090

GGUF Q4_K+Q6_K

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 17:47 UTCnewest release then: 0.9.0 (checked 2026-11-09)

76.4 tok/s

74% of the roofline103.8 tok/s possible

now, a token
13.08 ms
floor
9.63 ms
read, a token
8.65 GB
More
memory, measured
898 GB/s

What was run

llama-bench -m Qwen3-14B-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp77.1 tok/s of 104.6
512 tokens
llama.cpp76.4 tok/s of 103.8
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
13.08 ms
Roofline floor, a token
9.63 ms
Gap to the floor
3.45 ms
Kernels a token
281

Launch heavy: 766 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up Q4_K4.91 / 4.47 ms 91%
attn_qo Q4_K1.76 / 1.32 ms 75%
attention0.48 / 0.10 ms 20%
ffn_down Q6_K1.85 / 1.63 ms 88%
attn_kv Q4_K0.39 / 0.20 ms 50%
ffn_down Q4_K1.24 / 1.12 ms 90%
attn_kv Q6_K0.17 / 0.10 ms 58%
lm_head Q6_K0.75 / 0.71 ms 95%
norm0.69 ms · no floor
quantize0.41 ms · no floor
kernel gaps, at most0.36 ms · no floor
KV cache write0.07 ms · no floor
copy0.00 ms · no floor
elementwise0.00 ms · no floor
Per kernel (2): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q4_K10.18 / 7.10 ms ×240
mul_mat_vec_q · Q6_K3.01 / 2.44 ms ×41

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx3090-qwen3-14b-q4km-llama

Tested on

GPU
RTX 3090 · 24 GB
System RAM
252 GB
CPU threads
80
Memory, measured
898 GB/s
Temp, max
55 °C
Power
142 W mean of 280 W
SM clock
1788 MHz, throttled
Driver
595.71.05
Measured
2026-11-09 17:47 UTC · context 512 tokens · measured, in-model (nsys): 7 of 7 weight roles timed

Qwen3-14B on other GPUs