Qwen3-14B AWQ on RTX 5090
awq INT4_G128_ZP+BF16
Engines
vLLM0.31.0
153.0 tok/s
76% of the roofline200.9 tok/s possible
- now, a token
- 6.54 ms
- floor
- 4.98 ms
- read, a token
- 8.45 GB
Speed at context
128 tokens
512 tokens
Where the time goes
- Now, a token
- 6.54 ms
- Roofline floor, a token
- 4.98 ms
- Gap to the floor
- 1.56 ms
- Kernels a token
- –
Per role ms a token. floor (bytes ÷ measured memory speed) above the floor
This run timed the whole step only, so there is no per-role split.
No kernel capture for this run.
Run rtx5090-14b-vllm-batch
Tested on
- GPU
- RTX 5090 · 32 GB
- System RAM
- 31 GB
- CPU threads
- 16
- Memory, measured
- 1710 GB/s
- Temp, max
- 72 °C
- Power
- 232 W mean of 600 W
- SM clock
- 2085 MHz, throttled
- Driver
- 595.99.02
- Measured
- 2026-11-09 · context 512 tokens · measured, in-model (nsys): the whole step only, no per-role time
Qwen3-14B on other GPUs
RTX 5090consumer153.0 tok/s · 76%H100 80GB HBM3data center147.6 tok/s · 39%H200 NVLdata center135.0 tok/s · 26%RTX 4090consumer97.7 tok/s · 86%RTX 3090consumer82.4 tok/s · 78%L40Sdata center77.7 tok/s · 87%
Best engine and quant on each GPU, one user. The percentage is that GPU's own roofline.