boltbeam.fyi

2× RTX 3090 for local LLMs

3 measured runs. 24 GB of VRAM; memory measured at 896 GB/s, which sets the roofline for every model below.

ModelQuantEnginetok/sRooflineShare
Llama-3.3-70B-InstructAWQvLLM35.242.184%
Llama-3.3-70B-InstructQ4_K_Mllama.cpp19.221.390%
Qwen3.8-27BW4A16-AWQvLLM68.690.676%

Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.

Open the explorer on this GPU →