RTX 3090 for local LLMs
5 measured runs. 24 GB of VRAM; memory measured at 898 GB/s, which sets the roofline for every model below.
| Model | Quant | Engine | tok/s | Roofline | Share |
|---|---|---|---|---|---|
| Qwen3-14B | AWQ | vLLM | 82.4 | 105.4 | 78% |
| Qwen3-14B | Q4_K_M | llama.cpp | 76.4 | 103.8 | 74% |
| Qwen3-8B | Q4_K_M | llama.cpp | 124.5 | 188.9 | 66% |
| Qwen3.8-27B | W4A16-AWQ | vLLM | 46.2 | 57.6 | 80% |
| Qwen3.8-27B | UD-Q4_K_M | llama.cpp | 41.5 | 57.0 | 73% |
Share = measured tok/s over the roofline: the bytes a token reads over this GPU's measured memory speed. Open a quant for the test machine and the per-role and per-kernel breakdown.