boltbeam.fyi

RTX 3090 for local LLMs

5 measured runs. 24 GB of VRAM; memory measured at 898 GB/s, which sets the roofline for every model below.

ModelQuantEnginetok/sRooflineShare
Qwen3-14BAWQvLLM82.4105.478%
Qwen3-14BQ4_K_Mllama.cpp76.4103.874%
Qwen3-8BQ4_K_Mllama.cpp124.5188.966%
Qwen3.8-27BW4A16-AWQvLLM46.257.680%
Qwen3.8-27BUD-Q4_K_Mllama.cpp41.557.073%

Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.

Open the explorer on this GPU →