boltbeam.fyi

Qwen3.8-27B UD-Q4_K_M on RTX 5090

GGUF IQ4_XS+Q5_K

The tested file was 16.46 GB. The repo's current file is 16.56 GB: it was re-uploaded after the run.

Engines

llama.cpp

b11571 · 097cf33d9 ↗measured 2026-11-09 17:50 UTCnewest release then: 0.9.0 (checked 2026-11-09)

81.9 tok/s

75% of the roofline108.7 tok/s possible

now, a token
12.21 ms
floor
9.20 ms
read, a token
15.74 GB
More
memory, measured
1710 GB/s

What was run

llama-bench -m Qwen3.8-27B-UD-Q4_K_M.gguf -p 0 -n 33 -d 512 -ngl 99 -r 1 --no-warmup -o json -ub 512 -b 2048 -ctk f16 -ctv f16 -fa -1

Speed at context

128 tokens
llama.cpp82.1 tok/s of 108.8
512 tokens
llama.cpp81.9 tok/s of 108.7
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
12.21 ms
Roofline floor, a token
9.20 ms
Gap to the floor
3.00 ms
Kernels a token
1914

Launch heavy: 1914 kernel launches per token

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up IQ4_XS1.91 / 1.69 ms 89%
ssm_alpha_beta Q8_00.22 / 0.02 ms 7%
ffn_gate_up Q5_K1.12 / 1.00 ms 90%
ffn_gate_up Q4_K0.94 / 0.82 ms 88%
attention0.13 / 0.02 ms 16%
attn_qkv Q4_K0.59 / 0.50 ms 85%
ffn_down Q5_K0.87 / 0.79 ms 90%
ffn_down IQ4_XS0.74 / 0.67 ms 90%
ffn_gate_up Q3_K0.20 / 0.13 ms 67%
ssm_out Q5_K0.48 / 0.42 ms 87%
attn_gate Q4_K0.30 / 0.25 ms 83%
ffn_down Q4_K0.34 / 0.29 ms 86%
attn_qkv Q5_K0.29 / 0.25 ms 87%
attn_gate Q5_K0.18 / 0.15 ms 85%
attn_gate IQ4_XS0.14 / 0.12 ms 83%
attn_kv Q6_K0.05 / 0.03 ms 57%
attn_kv Q5_K0.04 / 0.02 ms 55%
ffn_down IQ3_S0.09 / 0.07 ms 79%
attn_qkv IQ4_XS0.12 / 0.10 ms 85%
attn_qo Q4_K0.14 / 0.12 ms 88%
attn_qo Q6_K0.07 / 0.06 ms 82%
attn_kv Q8_00.04 / 0.03 ms 67%
ffn_down IQ4_NL0.10 / 0.09 ms 88%
ssm_out Q6_K0.09 / 0.08 ms 87%
ffn_down Q3_K0.03 / 0.02 ms 67%
ssm_out Q4_K0.06 / 0.05 ms 83%
ffn_gate_up IQ4_NL0.10 / 0.09 ms 91%
ssm_out IQ4_XS0.06 / 0.05 ms 86%
ffn_down Q6_K0.05 / 0.04 ms 88%
ffn_gate_up IQ3_S0.03 / 0.02 ms 81%
lm_head Q6_K0.61 / 0.61 ms 99%
ffn_gate_up Q6_K0.05 / 0.04 ms 94%
attn_qkv IQ4_NL0.02 / 0.02 ms 87%
attn_kv Q4_K0.00 / 0.00 ms 49%
kernel gaps0.50 ms · no floor
quantize0.42 ms · no floor
norm0.36 ms · no floor
attn qo0.26 ms · no floor
copy0.23 ms · no floor
7 other small kernels0.57 ms · no floor
Per kernel (23): ms now / ms at the roofline, launches a token
mul_mat_vec_q · Q5_K3.21 ms · no floor ×122
mul_mat_vec_q · IQ4_XS3.15 ms · no floor ×97
mul_mat_vec_q · Q4_K2.37 / 2.04 ms ×98
mul_mat_vec_q · Q6_K0.92 / 0.86 ms ×24
quantize_q8_10.42 ms · no floor ×461
rms_norm_f320.36 ms · no floor ×305
mul_mat_vec_q · Q8_00.26 / 0.04 ms ×104
mul_mat_vec_q · Q3_K0.23 / 0.16 ms ×6
mul_mat_vec_q · IQ4_NL0.22 / 0.19 ms ×6
flash_attn_ext_f160.11 ms · no floor ×16
mul_mat_vec_q · IQ3_S0.11 / 0.09 ms ×4
gated_delta_net_cuda0.11 ms · no floor ×48
unary_gated_op_kernel0.09 ms · no floor ×140
k_get_rows_float_vec0.07 ms · no floor ×48
k_bin_bcast0.07 ms · no floor ×97
k_get_rows_float0.07 ms · no floor ×50
ssm_conv_f320.06 ms · no floor ×48
cpy_scalar0.05 ms · no floor ×64
concat_cont0.05 ms · no floor ×48
rope_multi0.04 ms · no floor ×32
unary_op_kernel0.03 ms · no floor ×48
k_set_rows0.02 ms · no floor ×32
flash_attn_stream_k_fixup_uniform0.01 ms · no floor ×16

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx5090-qwen3.8-27b-q4km-llama

Tested on

GPU
RTX 5090 · 32 GB
System RAM
31 GB
CPU threads
16
Memory, measured
1710 GB/s
Temp, max
60 °C
Power
156 W mean of 600 W
SM clock
2385 MHz
Driver
595.99.02
Measured
2026-11-09 17:50 UTC · context 512 tokens · measured, in-model (nsys): 33 of 39 weight roles timed

Qwen3.8-27B on other GPUs