Guides
Built from the measured runs on this site. Every number links to the run behind it.
- Which GPU is fastest for local LLMs? Measured tok/s, one userDecode speed of Qwen3-14B and Qwen3.8-27B on RTX 3090, 4090, 5090, L40S, H100 and H200, each against the GPU's measured roofline. An RTX 5090 decodes the 27B as fast as an H100.
- llama.cpp vs vLLM for one user: measured on 10 GPU and model pairsSame model, same GPU, one user: vLLM with AWQ against llama.cpp with Q4_K_M. vLLM led all 10 pairs, by 2 to 11%. The matrix kernels tie; the gap is the small kernels between them.
- What is a roofline? The fastest your GPU can run an LLMDecode speed for one user is capped by bytes per token over memory speed. How to compute the roofline, and why kernels fall short of it: Little's law and Volkov's latency hiding, with RTX 5090 examples.