boltbeam.fyi

Qwen3-14B AWQ on RTX 5090

awq INT4_G128_ZP+BF16

Engines

vLLM0.31.0

153.0 tok/s

76% of the roofline200.9 tok/s possible

now, a token
6.54 ms
floor
4.98 ms
read, a token
8.45 GB

Speed at context

128 tokens
vLLM156.2 tok/s of 202.4
512 tokens
vLLM153.0 tok/s of 200.9
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
6.54 ms
Roofline floor, a token
4.98 ms
Gap to the floor
1.56 ms
Kernels a token
–

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

No kernel capture for this run.

Run rtx5090-14b-vllm-batch

Tested on

GPU
RTX 5090 · 32 GB
System RAM
31 GB
CPU threads
16
Memory, measured
1710 GB/s
Temp, max
72 °C
Power
232 W mean of 600 W
SM clock
2085 MHz, throttled
Driver
595.99.02
Measured
2026-11-09 · context 512 tokens · measured, in-model (nsys): the whole step only, no per-role time

Qwen3-14B on other GPUs