4x A10G
4 records across 1 models (quants: EXL2 (bpw not stated)).
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 8.98 | – | – | – | – | 2024-07 | community | – | source |
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 8.76 | – | – | – | – | 2024-07 | community | – | source |
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 7.68 | – | – | – | 12500 | 2024-07 | community | – | source |
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 3.76 | – | – | – | 12500 | 2024-07 | community | – | source |
All rows cite their source. Provenance: community 40018 sourced 1085