boltbeam.fyi

Qwen3.8-27B W4A16-AWQ on H100 80GB HBM3

safetensors INT4_G128_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 17:43 UTCnewest release then: 0.33.0 (checked 2026-11-09)

87.8 tok/s

41% of the roofline211.9 tok/s possible

now, a token
11.39 ms
floor
4.72 ms
read, a token
15.25 GB
More
memory, measured
3237 GB/s

What was run

python vllm_decode_drive.py --model Qwen3.8-27B-W4A16-AWQ --contexts 512 --batches 1 --tokens 64 --reps 1 --once 64 --max-num-batched-tokens 584 --model-context-limit 584 --capture-pass paired

Speed at context

128 tokens
vLLM87.5 tok/s of 212.2
512 tokens
vLLM87.8 tok/s of 211.9
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
11.39 ms
Roofline floor, a token
4.72 ms
Gap to the floor
6.67 ms
Kernels a token
–

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

No kernel capture for this run.

Run qwen3-8-27b-w4a16-awq-nvidia_h100_80gb_hbm3_80g-001

Tested on

GPU
H100 80GB HBM3 · 80 GB
System RAM
443 GB
CPU threads
52
Memory, measured
3237 GB/s
Temp, max
38 °C
Power
141 W mean of 700 W
SM clock
1980 MHz
Driver
580.126.20
Measured
2026-11-09 17:43 UTC · context 512 tokens · measured, in-model (nsys): the whole step only, no per-role time

Qwen3.8-27B on other GPUs