boltbeam.fyi

H100 80GB for local LLMs

5 measured runs. 80 GB of VRAM; memory measured at 3252 GB/s, which sets the roofline for every model below.

ModelQuantEnginetok/sRooflineShare
Qwen3-14BAWQvLLM147.6380.339%
Qwen3-14BQ4_K_Mllama.cpp144.4374.339%
Qwen3-8BQ4_K_Mllama.cpp227.0684.433%
Qwen3.8-27BW4A16-AWQvLLM87.8211.941%
Qwen3.8-27BUD-Q4_K_Mllama.cpp83.6205.741%

Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.

Open the explorer on this GPU →