boltbeam.fyi

RTX 5090 for local LLMs

6 measured runs. 32 GB of VRAM; memory measured at 1710 GB/s, which sets the roofline for every model below.

ModelQuantEnginetok/sRooflineShare
Qwen3-14BAWQvLLM153.0200.976%
Qwen3-14BQ4_K_Mllama.cpp149.5197.876%
Qwen3.8-27BW4A16-AWQvLLM87.5109.780%
Qwen3.8-27BUD-Q4_K_Mllama.cpp81.9108.775%
Qwen3.8-Flash-NextGSQ-RCO IQ1_Mllama.cpp + mods226.1528.543%
Qwen3.8-Flash-NextGSQ-RCO IQ1_Mllama.cpp77.2537.114%

Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.

Open the explorer on this GPU →