boltbeam.fyi

Qwen3-14B AWQ on H100 80GB

awq INT4_G128_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 18:20 UTCnewer release at the time: 0.33.0

147.6 tok/s

39% of the roofline380.3 tok/s possible

now, a token
6.78 ms
floor
2.63 ms
read, a token
8.45 GB
More
memory, measured
3237 GB/s

What was run

python vllm_decode_drive.py --model Qwen3-14B-AWQ --contexts 512 --batches 1 --tokens 64 --reps 1 --once 64 --max-num-batched-tokens 584 --model-context-limit 584 --capture-pass paired

Speed at context

128 tokens
vLLM148.4 tok/s of 383.1
512 tokens
vLLM147.6 tok/s of 380.3
The pink tick is the roofline at that context.

Where the time goes

Now, a token
6.78 ms
Roofline floor, a token
2.63 ms
Gap to the floor
4.15 ms
Kernels a token
–

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

No kernel capture for this run.

Run h100-qwen3-14b-awq-vllm

Tested on

GPU
H100 80GB · 80 GB
System RAM
443 GB
CPU threads
52
Memory, measured
3237 GB/s
Temp, max
39 °C
Power
143 W mean of 700 W
SM clock
1979 MHz
Driver
580.126.20
Measured
2026-11-09 18:20 UTC · context 512 tokens

Qwen3-14B on other GPUs