Llama3 fine-tuned
4 records across 1 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 8.98 | – | – | – | – | 2024-07 | community | – | source |
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 8.76 | – | – | – | – | 2024-07 | community | – | source |
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 7.68 | – | – | – | 12500 | 2024-07 | community | – | source |
| 4x A10G | Llama3 fine-tuned 70B | EXL2 (bpw not stated) | ExLlamaV2 | 3.76 | – | – | – | 12500 | 2024-07 | community | – | source |