boltbeam.fyi

Nemotron-3.5-Lightning-30B NVFP4 on RTX 5090

safetensors NVFP4_G16+FP8_E4M3

Engines

vLLM

0.31.0 ↗measured 2026-11-10 03:34 UTCnewer release at the time: 0.33.0

378.9 tok/s

55% of the roofline686.3 tok/s possible

now, a token
2.64 ms
floor
1.46 ms
read, a token
2.49 GB

Speed at context

128 tokens
vLLM377.5 tok/s
512 tokens
vLLM378.9 tok/s of 686.3
The pink tick is the roofline at that context.

Where the time goes

Now, a token
2.64 ms
Roofline floor, a token
1.46 ms
Gap to the floor
1.18 ms
Kernels a token
534

Per role ms a token. floor (bytes ÷ measured memory speed) above the floor

moe_shared_expert_down NVFP4_G160.39 / 0.08 ms 19%
ssm_out FP8_E4M30.29 / 0.15 ms 51%
moe_expert_down NVFP4_G160.34 / 0.23 ms 66%
moe_expert_gate_up NVFP4_G160.33 / 0.23 ms 69%
ssm_in FP8_E4M30.47 / 0.37 ms 80%
attention0.09 / 0.00 ms 1%
moe_shared_expert_gate_up NVFP4_G160.16 / 0.08 ms 46%
mamba2 state_update0.08 / 0.03 ms 38%
mamba2 convolution0.04 / 0.00 ms 3%
moe_router F320.05 / 0.02 ms 38%
attn_kv + attn_qo BF160.11 / 0.09 ms 76%
attn_qo BF160.09 / 0.08 ms 83%
lm_head NVFP4_G160.13 / 0.12 ms 92%
kernel gaps0.08 ms · no floor
vllm::moe::single_group_topk::detail::s…0.08 ms · no floor
vllm::moe::moe_align_block_size_kernel0.07 ms · no floor
at::native::vectorized_elementwise_kern…0.06 ms · no floor
sm89_xmma_gemm_e4m3bf16_e4m3f32_f32_tn_…0.05 ms · no floor
26 other small kernels-0.26 ms · no floor
Per kernel (40): ms now / ms at the roofline, launches a token
Marlin0.68 / 0.27 ms ×47
marlin_moe_wna16::Marlin0.67 / 0.45 ms ×46
sm89_xmma_gemm_e4m3bf16_e4m3f32_f32_tn_n_tilesize64x128x64_stage4_warpsize2x2x1_tensor16x8x32_execute_kernel__5x_cublas0.47 / 0.37 ms ×23
nvjet_sm120_qqtst_mma_64x32x64_16_32x16x64_tmaAB_bz_TNNN0.29 / 0.15 ms ×23
gemm · attn_kv + attn_qo BF160.11 / 0.09 ms ×6
gemm · attn_qo BF160.09 / 0.08 ms ×6
kernel_mha0.09 / 0.00 ms ×6
vllm::moe::single_group_topk::detail::s…0.08 ms · no floor ×23
_selective_scan_update_kernel0.08 / 0.03 ms ×23
vllm::moe::moe_align_block_size_kernel0.07 ms · no floor ×23
at::native::vectorized_elementwise_kern…0.06 ms · no floor ×49
sm89_xmma_gemm_e4m3bf16_e4m3f32_f32_tn_…0.05 ms · no floor ×23
triton_per_fused__to_copy_add_clamp_mea…0.05 ms · no floor ×23
dot_kernel0.05 / 0.02 ms ×23
triton_red_fused__to_copy_add_clamp_fus…0.05 ms · no floor ×22
_causal_conv1d_update_kernel0.04 / 0.00 ms ×23
triton_red_fused__to_copy_fused_add_rms…0.03 ms · no floor ×23
triton_poi_fused_pow_relu_00.02 ms · no floor ×23
vllm::moe::moe_sum_vec_kernel0.02 ms · no floor ×23
reduce_1Block_kernel0.02 ms · no floor ×23
vllm::moe::count_and_sort_expert_tokens…0.02 ms · no floor ×23
vllm::reshape_and_cache_flash_kernel0.02 ms · no floor ×6
_gumbel_sample_kernel0.01 ms · no floor ×1
triton_red_fused_fused_add_rms_norm_vie…0.01 ms · no floor ×6
at::native::index_elementwise_kernel0.01 ms · no floor ×3
_bias_kernel0.00 ms · no floor ×1
triton_red_fused__to_copy_clamp_embeddi…0.00 ms · no floor ×1
_gather_block_tables_kernel0.00 ms · no floor ×1
at::native::unrolled_elementwise_kernel0.00 ms · no floor ×1
at::native::reduce_kernel0.00 ms · no floor ×1

No floor = the kernel reads no weights (norms, conversions, copies): all of its time is gap.

Run rtx5090-nemotron-3.5-lightning-30b-nvfp4-vllm-i

Tested on

GPU
RTX 5090 · 32 GB
System RAM
31 GB
CPU threads
16
Memory, measured
1710 GB/s
Temp, max
see notes
Power
see notes
SM clock
–
Driver
595.99.02
Measured
2026-11-10 03:34 UTC · context 512 tokens · measured, in-model (nsys): split by role from the engine's own labels, 10 weight roles

Nemotron-3.5-Lightning-30B on other GPUs

Nemotron-3.5-Lightning-30B has only been measured on the RTX 5090 so far.