boltbeam.fyi

Qwen3-14B AWQ on L40S

awq INT4_G128_ZP+BF16

Engines

vLLM

0.31.0 ↗measured 2026-11-09 19:45 UTCnewest release then: 0.33.0 (checked 2026-11-09)

77.7 tok/s

87% of the roofline89.4 tok/s possible

now, a token
12.87 ms
floor
11.18 ms
read, a token
8.52 GB
More
memory, measured
762 GB/s

What was run

python vllm_decode_drive.py --model Qwen3-14B-AWQ --contexts 512 --batches 1 --tokens 64 --reps 1 --once 64 --max-num-batched-tokens 584 --model-context-limit 584 --capture-pass paired

Speed at context

128 tokens
vLLM79.1 tok/s
512 tokens
vLLM77.7 tok/s of 89.4
Measured at short contexts only. The pink tick is the roofline at that context.

Where the time goes

Now, a token
12.87 ms
Roofline floor, a token
11.18 ms
Gap to the floor
1.69 ms
Kernels a token
460

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

ffn_gate_up INT4_G128_ZP5.10 / 4.87 ms 95%
ffn_down INT4_G128_ZP2.61 / 2.43 ms 93%
attn_qo INT4_G128_ZP0.89 / 0.72 ms 81%
attention0.28 / 0.12 ms 42%
attn_kv + attn_qo INT4_G128_ZP1.15 / 1.00 ms 87%
lm_head F162.16 / 2.04 ms 95%
flash::flash_fwd_splitkv_combine_kernel0.14 ms · no floor
triton_red_fused_30.11 ms · no floor
kernel gaps0.09 ms · no floor
triton_red_fused_fused_add_rms_norm_20.09 ms · no floor
triton_red_fused_fused_add_rms_norm_00.08 ms · no floor
20 other small kernels0.18 ms · no floor
Per kernel (27): ms now / ms at the roofline, launches a token
Marlin9.75 / 9.02 ms ×160
gemm · lm_head F162.16 / 2.04 ms ×1
flash_fwd_splitkv_kernel0.28 / 0.12 ms ×40
flash::flash_fwd_splitkv_combine_kernel0.14 ms · no floor ×40
triton_red_fused_30.11 ms · no floor ×39
triton_red_fused_fused_add_rms_norm_20.09 ms · no floor ×40
triton_red_fused_fused_add_rms_norm_00.08 ms · no floor ×40
vllm::reshape_and_cache_flash_kernel0.07 ms · no floor ×40
triton_poi_fused_mul_silu_slice_10.05 ms · no floor ×40
at::native::index_elementwise_kernel0.01 ms · no floor ×3
_gumbel_sample_kernel0.01 ms · no floor ×1
_bias_kernel0.01 ms · no floor ×1
_gather_block_tables_kernel0.00 ms · no floor ×1
at::native::reduce_kernel0.00 ms · no floor ×1
triton_red_fused__to_copy_embedding_rms…0.00 ms · no floor ×1
_combine_sampled_and_draft_tokens_kernel0.00 ms · no floor ×1
_post_update_kernel0.00 ms · no floor ×1
_get_num_sampled_and_rejected_kernel0.00 ms · no floor ×1
at::native::vectorized_elementwise_kern…0.00 ms · no floor ×3
_compute_slot_mappings_kernel0.00 ms · no floor ×1
at::native::unrolled_elementwise_kernel0.00 ms · no floor ×1
triton_red_fused_10.00 ms · no floor ×1
at::native::_scatter_gather_elementwise…0.00 ms · no floor ×1
_prepare_pos_seq_lens_kernel0.00 ms · no floor ×1
<unnamed>::elementwise_kernel_with_index0.00 ms · no floor ×1
_apply_write_kernel0.00 ms · no floor ×0
concurrent kernel overlap (subtract)-0.00 / -0.00 ms ×0

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run qwen3-14b-awq-nvidia_l40s_45g-001

Tested on

GPU
L40S · 45 GB
System RAM
2016 GB
CPU threads
192
Memory, measured
762 GB/s
Temp, max
56 °C
Power
125 W mean of 350 W
SM clock
2510 MHz, throttled
Driver
570.211.01
Measured
2026-11-09 19:45 UTC · context 512 tokens · measured, in-model (nsys): split by role from the engine's own labels, 5 weight roles

Qwen3-14B on other GPUs