Qwen3 4B
65 records across 9 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| B300 | Qwen3 4B | BF16 | vLLM | 10543 | – | – | – | tg256 | 2026-10-02 | community | – | source |
| B300 | Qwen3 4B | BF16 | vLLM | 10495 | – | – | – | tg256 | 2026-10-02 | community | – | source |
| B300 | Qwen3 4B | BF16 | vLLM | 3032 | – | – | – | tg256 | 2026-10-02 | community | – | source |
| B300 | Qwen3 4B | BF16 | vLLM | 3012 | – | – | – | tg256 | 2026-10-02 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 656.60 | – | – | – | tg512 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 607.98 | – | – | – | tg1024 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 469.62 | – | – | – | tg512 | 2025-05 | community | – | source |
| B300 | Qwen3 4B | BF16 | vLLM | 464.7 | – | – | – | tg256 | 2026-10-02 | community | – | source |
| B300 | Qwen3 4B | BF16 | vLLM | 462.3 | – | – | – | tg256 | 2026-10-02 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 433.77 | – | – | – | tg512 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 424.58 | – | – | – | tg512 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 423.26 | – | – | – | tg1024 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 413.54 | – | – | – | tg512 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 401.26 | – | – | – | tg512 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 395.12 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 377.98 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 369.19 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 337.17 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 278.73 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | vLLM | 249.99 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 239.82 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | AWQ | SGLang | 238.21 | – | – | – | tg3096 | 2025-05 | community | – | source |
| A10 | Qwen3 4B | BF16 | vLLM | 138.3 | – | – | – | – | 2026-10-01 | community | – | source |
| A10 | Qwen3 4B | BF16 | vLLM | 132.6 | – | – | – | – | 2026-10-01 | community | – | source |
| M5 Max | Qwen3 4B | 8bit | oMLX | 90.0 | 4528.0 | – | – | 8192 | 2026-10-11 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 82.8 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 81.6 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 80.5 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 79.7 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 78.2 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 77.0 | – | – | – | – | 2026-10-01 | community | – | source |
| M5 Max | Qwen3 4B | 8bit | oMLX | 74.5 | 3844.0 | – | – | 16384 | 2026-10-11 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 61.2 | – | – | – | – | 2026-10-01 | community | – | source |
| M5 Max | Qwen3 4B | 8bit | oMLX | 52.0 | 2643.0 | – | – | 32768 | 2026-10-11 | community | – | source |
| M5 Max | Qwen3 4B | 8bit | oMLX | 52.0 | 2643.0 | – | – | 32768 | 2026-10-11 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 39.2 | – | – | – | – | 2026-10-01 | community | contradiction | source |
| M5 Max | Qwen3 4B | 8bit | oMLX | 35.2 | 1352.0 | – | – | 65536 | 2026-10-11 | community | – | source |
| M5 Max | Qwen3 4B | 8bit | oMLX | 35.2 | 1352.0 | – | – | 65536 | 2026-10-11 | community | – | source |
| M4 | Qwen3 4B | 4bit | oMLX | 29.4 | 305.8 | – | – | 4096 | 2026-10-08 | community | – | source |
| K12-1 Ryzen H255 64gb | Qwen3 4B | Q8_0 | Vulkan | 18.040894 | 393.674985 | – | – | – | 2026-09 | community | – | source |
| M4 | Qwen3 4B | 4bit | oMLX | 16.2 | 211.8 | – | – | 16384 | 2026-10-08 | community | – | source |
| 2 GMKtec K12 | Qwen3 4B | Q8_0 | llama.cpp (RPC) | 13.211557 | 66.181528 | – | – | – | 2026-09 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | 11.48 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | 11.48 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | 11.32 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | 11.22 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | 10.92 | – | – | – | 128 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | 10.78 | – | – | – | 128 | 2026-10 | community | – | source |
| 2 GMKtec K12 | Qwen3 4B | Q8_0 | llama.cpp (RPC) | 5.411568 | 63.902555 | – | – | – | 2026-09 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 95.95 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 310.88 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 176.81 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 552.12 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 307.15 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 810.96 | – | – | 2048 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 87.88 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 237.88 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 162.5 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 430.49 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 284.02 | – | – | 4096 | 2026-10 | community | – | source |
| AMD Ryzen 9 9950X | Qwen3 4B | Q8_0 | llama.cpp (CPU) | – | 657.98 | – | – | 4096 | 2026-10 | community | – | source |
| Arc Pro B70 | Qwen3 4B | Q8_0 | llama.cpp | – | 2985.5 | – | – | 512 | 2026-10-17 | sourced | – | source |
| Arc Pro B70 | Qwen3 4B | Q8_0 | llama.cpp | – | 4831.58 | – | – | 512 | 2026-10-17 | sourced | – | source |
| Arc Pro B70 | Qwen3 4B | Q8_0 | llama.cpp | – | 5489.1 | – | – | 512 | 2026-10-18 | sourced | – | source |
| Arc Pro B70 | Qwen3 4B | Q8_0 | llama.cpp | – | 5652.1 | – | – | 512 | 2026-10-18 | sourced | – | source |