boltbeam.fyi

Qwen3.8-27B W4A16-AWQ on 2× RTX 3090

safetensors INT4_G32_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 02:15 UTC

68.6 tok/s

76% of the roofline90.6 tok/s possible

now, a token
14.58 ms
floor
11.04 ms
read, a token
9.87 GB

Speed at context

128 tokens
vLLM68.8 tok/s of 90.8
512 tokens
vLLM68.6 tok/s of 90.6
The pink tick is the roofline at that context.

Where the time goes

Now, a token
14.58 ms
Roofline floor, a token
11.04 ms
Gap to the floor
3.54 ms
Kernels a token
–

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

This run timed the whole step only, so there is no per-role split.

No kernel capture for this run.

Run 3090x2-vllm-27b-awq-row

Tested on

GPU
2× RTX 3090 · 24 GB each
System RAM
63 GB
CPU threads
8
Memory, measured
896 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.112.07
Measured
2026-11-09 02:15 UTC · context 512 tokens · measured, in-model (nsys): the whole step only, no per-role time

Qwen3.8-27B on other GPUs