L4
18 records across 2 models (quants: BF16, PQ2_0, PTQ1_0, Ternary PQ2_0, Ternary PTQ1_0).
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| L4 | Qwen3 4B | BF16 | vLLM | 82.8 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 81.6 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 80.5 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 79.7 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 78.2 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 77.0 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 61.2 | – | – | – | – | 2026-10-01 | community | – | source |
| L4 | Qwen3 4B | BF16 | vLLM | 39.2 | – | – | – | – | 2026-10-01 | community | contradiction | source |
| L4 | Ternary-Bonsai-2-27B | PQ2_0 | llama.cpp (PrismML fork) | 68.4 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PQ2_0 | llama.cpp (PrismML fork) | 67.8 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PQ2_0 | llama.cpp (PrismML fork) | 67.6 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PQ2_0 | llama.cpp (PrismML fork) | 42.9 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | Ternary PTQ1_0 | llama.cpp (CUDA) | 32.1 | 467 | – | – | tg128 | 2026-09-16 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PTQ1_0 | llama.cpp (PrismML fork) | 31.6 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PTQ1_0 | llama.cpp (PrismML fork) | 31.4 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PTQ1_0 | llama.cpp (PrismML fork) | 31.3 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | PTQ1_0 | llama.cpp (PrismML fork) | 30.6 | – | – | – | – | 2026-09 | community | – | source |
| L4 | Ternary-Bonsai-2-27B | Ternary PQ2_0 | llama.cpp (CUDA) | 29.8 | 777 | – | – | tg128 | 2026-09-16 | community | – | source |
All rows cite their source. Provenance: community 40018 sourced 1085