Qwen2.5-7B
19 records across 5 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A10 | Qwen2.5-7B | AWQ | SGLang | 1544.94 | – | – | – | 8192 | 2025-04 | community | – | source |
| 2 A10 | Qwen2.5-7B | AWQ | SGLang | 1157.76 | – | – | – | 8192 | 2025-04 | community | – | source |
| 2 A10 | Qwen2.5-7B | AWQ | SGLang | 1151.33 | – | – | – | 8192 | 2025-04 | community | – | source |
| 2 A10 | Qwen2.5-7B | AWQ | vLLM | 1074.01 | – | – | – | 8192 | 2025-04 | community | – | source |
| 2 A10 | Qwen2.5-7B | AWQ | vLLM | 1025.81 | – | – | – | 8192 | 2025-04 | community | – | source |
| A10 | Qwen2.5-7B | AWQ | vLLM | 961.05 | – | – | – | 8192 | 2025-04 | community | – | source |
| 2 A10 | Qwen2.5-7B | AWQ | SGLang | 853.94 | – | – | – | 8192 | 2025-04 | community | – | source |
| 2 A10 | Qwen2.5-7B | AWQ | SGLang | 511.92 | – | – | – | 8192 | 2025-04 | community | – | source |
| A10 | Qwen2.5-7B | AWQ | SGLang | 333.00 | – | – | – | 8192 | 2025-04 | community | – | source |
| A10 | Qwen2.5-7B | AWQ | vLLM | 259.53 | – | – | – | 8192 | 2025-04 | community | – | source |
| RTX 4090 | Qwen2.5-7B | Q4_0 | llama.cpp (CUDA) | 163.51 | – | – | – | 4096 | 2026-10-05 | community | – | source |
| RTX 4090 | Qwen2.5-7B | Q4_0 | llama.cpp (CUDA) | 149.39 | – | – | – | 4096 | 2026-10-05 | community | – | source |
| RTX 4090 | Qwen2.5-7B | Q4_0 | llama.cpp (CUDA) | 136.26 | – | – | – | 16384 | 2026-10-05 | community | – | source |
| 2x RX 7900 XTX | Qwen2.5-7B | Q8_0 | llama.cpp | 122.63 | – | – | – | tg512 | 2026-09 | community | – | source |
| RTX 4090 | Qwen2.5-7B | Q4_0 | llama.cpp (CUDA) | 110.98 | – | – | – | 32768 | 2026-10-05 | community | – | source |
| RTX 4090 | Qwen2.5-7B | Q4_0 | llama.cpp (CUDA) | 110.87 | – | – | – | 16384 | 2026-10-05 | community | – | source |
| RX 7900 XTX | Qwen2.5-7B | Q8_0 | llama.cpp | 95.9 | 4534.64 | – | – | tg512 | 2026-09 | community | – | source |
| RX 7900 XTX | Qwen2.5-7B | Q8_0 | llama.cpp | 95.2 | 4308.07 | – | – | tg512 | 2026-09 | community | – | source |
| RTX 4090 | Qwen2.5-7B | Q4_0 | llama.cpp (CUDA) | 88.07 | – | – | – | 32768 | 2026-10-05 | community | – | source |