Cross-source checks
Where a measured llama.cpp row (any source) and the LLM Configurator bandwidth-model estimate cover the same model + chip at Q4_K_M / 4096 ctx, they are shown side by side. 5 strict overlaps so far.
| model | hardware | measured tok/s | source | estimated tok/s | delta | class |
|---|---|---|---|---|---|---|
| qwen-2.5-7b | M3 Max | 40 | source | 40 | +0.0% | in-sample (fitted) |
| qwen-2.5-7b | RTX 3090 | 95 | source | 95 | +0.0% | in-sample (fitted) |
| qwen-2.5-7b | RTX 4090 | 135 | source | 135 | +0.0% | in-sample (fitted) |
| Qwen3 8B | RTX 4090 | 141.3 | source | 141.3 | +0.0% | in-sample (fitted) |
| Qwen3 8B | RTX 5090 | 212.66 | source | 169.7 | -20.2% | out-of-sample |
Q4-K family reference (measured quant differs from the estimate's)
Same model + chip + 4096 ctx, but the measured row is another Q4-K quant; the quant difference contributes part of the delta.
| model | hardware | measured quant | measured tok/s | source | estimated tok/s | delta | class |
|---|---|---|---|---|---|---|---|
| Llama 3.1 8B | RTX 4090 | Q4_K_XL | 131 | source | 125.16 | -4.5% | reference |
| Qwen3 14B | RTX 3080 Ti | Q4_K | 69.9 | source | 52.16 | -25.4% | reference |
| Qwen3 14B | RTX 3090 Ti | Q4_K | 76.2 | source | 56.95 | -25.3% | reference |
| Qwen3 14B | RTX 4060 Ti 16GB | Q4_K | 27.4 | source | 26.16 | -4.5% | reference |
| Qwen3 14B | RTX 4070 | Q4_K | 42.5 | source | 43.84 | +3.2% | reference |
| Qwen3 14B | RTX 4070 SUPER | Q4_K | 45.5 | source | 43.84 | -3.6% | reference |
| Qwen3 14B | RTX 4070 Ti | Q4_K | 45.8 | source | 43.84 | -4.3% | reference |
| Qwen3 14B | RTX 4070 Ti SUPER | Q4_K | 58.1 | source | 56.6 | -2.6% | reference |
| Qwen3 14B | RTX 4080 | Q4_K | 62.0 | source | 59.88 | -3.4% | reference |
| Qwen3 14B | RTX 4080 SUPER | Q4_K | 64.2 | source | 61.25 | -4.6% | reference |
| Qwen3 14B | RTX 4090 | Q4_K | 84.4 | source | 79.83 | -5.4% | reference |
| Qwen3 14B | RTX 5060 Ti 16GB | Q4_K | 41.1 | source | 36.4 | -11.4% | reference |
| Qwen3 14B | RTX 5070 | Q4_K | 54.2 | source | 52.45 | -3.2% | reference |
| Qwen3 14B | RTX 5070 Ti | Q4_K | 74.3 | source | 67.3 | -9.4% | reference |
| Qwen3 14B | RTX 5080 | Q4_K | 80.6 | source | 71.33 | -11.5% | reference |
| Qwen3 14B | RTX 5090 | Q4_K | 123.8 | source | 116.92 | -5.6% | reference |
| Qwen3 30B A3B | RTX 3090 Ti | Q4_K | 166.9 | source | 164.9 | -1.2% | reference |
| Qwen3 30B A3B | RTX 4090 | Q4_K | 207.3 | source | 207.94 | +0.3% | reference |
| Qwen3 8B | RTX 3080 Ti | Q4_K | 115.2 | source | 83.1 | -27.9% | reference |
| Qwen3 8B | RTX 3090 Ti | Q4_K | 123.7 | source | 90.08 | -27.2% | reference |
| Qwen3 8B | RTX 4060 Ti 16GB | Q4_K | 45.8 | source | 43.38 | -5.3% | reference |
| Qwen3 8B | RTX 4070 | Q4_K | 71.2 | source | 70.75 | -0.6% | reference |
| Qwen3 8B | RTX 4070 SUPER | Q4_K | 75.4 | source | 70.75 | -6.2% | reference |
| Qwen3 8B | RTX 4070 Ti | Q4_K | 75.8 | source | 70.75 | -6.7% | reference |
| Qwen3 8B | RTX 4070 Ti SUPER | Q4_K | 96.3 | source | 89.58 | -7.0% | reference |
| Qwen3 8B | RTX 4080 | Q4_K | 102.7 | source | 94.31 | -8.2% | reference |
| Qwen3 8B | RTX 4080 SUPER | Q4_K | 104.2 | source | 96.27 | -7.6% | reference |
| Qwen3 8B | RTX 4090 | Q4_K | 141.3 | source | 141.3 | +0.0% | reference |
| Qwen3 8B | RTX 5060 Ti 16GB | Q4_K | 69.2 | source | 59.41 | -14.1% | reference |
| Qwen3 8B | RTX 5070 | Q4_K | 85.8 | source | 83.54 | -2.6% | reference |
| Qwen3 8B | RTX 5070 Ti | Q4_K | 120.5 | source | 104.83 | -13.0% | reference |
| Qwen3 8B | RTX 5080 | Q4_K | 129.1 | source | 110.46 | -14.4% | reference |
Strict-overlap rows differ by at most 20.2%; in-sample configurations (the model was fitted from those runs) agree by construction. Estimated values: LLM Configurator benchmark cells (CC BY 4.0), calibrated per GPU architecture from the 14 measured runs in their published dataset. Delta = (estimated - measured) / measured.