llama-3-2-1b-instruct
65 records across 64 hardware.
| hardware | model | quant | backend | tok/s | pp tok/s | W | ttft s | ctx/tg | date | class | flags | source |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| M5 Max (32-core GPU, 36 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 228.99372777777776 | 3620.6608126666665 | – | 0.32193552311111107 | – | – | sourced | – | source |
| M4 Max (40-core GPU, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 182.5577164513889 | 3833.843851847222 | – | 0.30469161659027777 | – | – | sourced | – | source |
| M4 Max (40-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 180.24040433333337 | 3857.499544666666 | – | 0.30436414811111107 | – | – | sourced | – | source |
| M4 Max (40-core GPU, 48 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 178.9852262222222 | 3750.568106722222 | – | 0.30928531144444443 | – | – | sourced | – | source |
| M3 Ultra (80-core GPU, 512 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 178.83792144444442 | 5600.99501711111 | – | 0.20634095138888886 | – | – | sourced | – | source |
| M3 Ultra (80-core GPU, 256 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 177.900228 | 4999.496534111112 | – | 0.22653121744444446 | – | – | sourced | – | source |
| M2 Ultra (60-core GPU, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 176.3882528888889 | 3295.522693222222 | – | 0.33357317588888885 | – | – | sourced | – | source |
| M2 Ultra (60-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 174.08661300000003 | 3290.1508422222223 | – | 0.3351846761111111 | – | – | sourced | – | source |
| M2 Ultra (60-core GPU, 192 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 169.76082655555555 | 3272.209784333334 | – | 0.3388111018888889 | – | – | sourced | – | source |
| M4 Max (32-core GPU, 36 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 166.45011455555556 | 3268.880837361111 | – | 0.366560225611111 | – | – | sourced | – | source |
| M4 Max (GPU count not published, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 156.3000006666667 | 679.7101832222222 | – | 1.9661574258888888 | – | – | sourced | – | source |
| M2 Max (38-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 152.97702255555558 | 2551.365566666667 | – | 0.44196865744444436 | – | – | sourced | – | source |
| M1 Ultra (64-core GPU, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 151.0678697777778 | 2977.086063277778 | – | 0.3675812592777778 | – | – | sourced | – | source |
| M3 Max (40-core GPU, 48 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 148.95889516666665 | 3399.1241412222225 | – | 0.34880203711111113 | – | – | sourced | – | source |
| M3 Max (40-core GPU, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 146.33974311111112 | 3291.9091724444447 | – | 0.3584107959999999 | – | – | sourced | – | source |
| M1 Ultra (48-core GPU, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 137.97023270370372 | 2582.382230222222 | – | 0.4251138565185186 | – | – | sourced | – | source |
| M3 Max (30-core GPU, 36 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 132.9925948888889 | 2553.2959846666668 | – | 0.4669405787777778 | – | – | sourced | – | source |
| M3 Max (30-core GPU, 96 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 132.87379466666664 | 2519.9553065555556 | – | 0.4758228193333332 | – | – | sourced | – | source |
| M2 Max (30-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 127.58507908333334 | 2031.665092888889 | – | 0.5575502119166668 | – | – | sourced | – | source |
| M1 Max (32-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 125.77495338888887 | 2025.1921614074072 | – | 0.5596487252407406 | – | – | sourced | – | source |
| M1 Max (32-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 120.67175636111112 | 1972.074889361111 | – | 0.5782512951666667 | – | – | sourced | – | source |
| M2 Ultra (GPU count not published, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 120.4092842222222 | 659.8306766666667 | – | 1.9676880275555553 | – | – | sourced | – | source |
| M4 Pro (20-core GPU, 24 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 119.24874207407407 | 2128.7541195555555 | – | 0.5725979937407406 | – | – | sourced | – | source |
| M4 Pro (20-core GPU, 48 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 118.8963297948718 | 2134.135712188034 | – | 0.5720378254615385 | – | – | sourced | – | source |
| M4 Pro (20-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 118.5574338888889 | 2145.125925888889 | – | 0.5718198889722222 | – | – | sourced | – | source |
| M4 Pro (16-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 111.93509922222222 | 1858.9418227777776 | – | 0.6703511943333332 | – | – | sourced | – | source |
| M4 Pro (16-core GPU, 48 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 111.00253472222222 | 1754.577648111111 | – | 0.6816085969999999 | – | – | sourced | – | source |
| M4 Pro (16-core GPU, 24 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 110.94556159259258 | 1823.817922962963 | – | 0.6715746882962962 | – | – | sourced | – | source |
| M3 Max (40-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 106.97443855555557 | 2521.617585666667 | – | 0.520643525388889 | – | – | sourced | – | source |
| M1 Max (24-core GPU, 64 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 105.66463296296295 | 1669.4850333333334 | – | 0.6893040093333332 | – | – | sourced | – | source |
| M2 Pro (19-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 100.2557703111111 | 1487.5642871333334 | – | 0.8063303454888889 | – | – | sourced | – | source |
| M2 Pro (19-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 99.52299688888888 | 1457.8697797777777 | – | 0.8091630742222221 | – | – | sourced | – | source |
| M5 (10-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 98.37004172222224 | 1271.5481898333337 | – | 0.989687108888889 | – | – | sourced | – | source |
| M5 (10-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 98.08477433333331 | 1244.9073107777776 | – | 0.995430389 | – | – | sourced | – | source |
| M1 Max (24-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 93.859194 | 1582.032366 | – | 0.7243009050000001 | – | – | sourced | – | source |
| M2 Pro (16-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 91.45281944444444 | 1281.403522111111 | – | 0.9141989862222222 | – | – | sourced | – | source |
| M2 Pro (16-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 91.14859188888887 | 1328.2018266666669 | – | 0.9017461759444444 | – | – | sourced | – | source |
| M3 Pro (18-core GPU, 36 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 89.75982788888888 | 1586.168225333333 | – | 0.8094999791666667 | – | – | sourced | – | source |
| M3 Pro (14-core GPU, 36 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 88.23620972222221 | 1327.8959733333334 | – | 0.9533793772777779 | – | – | sourced | – | source |
| M3 Pro (14-core GPU, 18 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 88.10604988888889 | 1343.9570948888888 | – | 0.9594814676666664 | – | – | sourced | – | source |
| M3 Pro (18-core GPU, 18 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 85.63227766666667 | 1573.6998116666664 | – | 0.8258373006666666 | – | – | sourced | – | source |
| M1 Pro (16-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 78.15808337037036 | 1158.147661425926 | – | 1.0274150409074074 | – | – | sourced | – | source |
| M1 Pro (16-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 77.23935995555556 | 1165.9555193555557 | – | 1.0365038463555556 | – | – | sourced | – | source |
| M4 (10-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 76.16243562222222 | 1091.1117940111112 | – | 1.1983527194666668 | – | – | sourced | – | source |
| M4 (10-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 75.56749613888888 | 1069.8806100277775 | – | 1.2130988726666667 | – | – | sourced | – | source |
| M4 (10-core GPU, 24 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 75.4330094074074 | 1035.9960293703705 | – | 1.2527001605185184 | – | – | sourced | – | source |
| M1 Pro (14-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 71.81051661111111 | 1040.067598027778 | – | 1.1483909050555556 | – | – | sourced | – | source |
| M1 Pro (14-core GPU, 32 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 71.01946466666666 | 1063.8479185555557 | – | 1.1336526296666667 | – | – | sourced | – | source |
| M4 (GPU count not published, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 67.95033205555555 | 238.95714649999996 | – | 5.743805437444445 | – | – | sourced | – | source |
| M3 (10-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 67.18826562222223 | 931.5106880888889 | – | 1.4093825610888888 | – | – | sourced | – | source |
| M4 (8-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 65.86037953333333 | 897.0363834222222 | – | 1.4411192250444445 | – | – | sourced | – | source |
| M3 (10-core GPU, 24 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 64.71307648888889 | 916.5074562888888 | – | 1.4258484739777777 | – | – | sourced | – | source |
| M3 (GPU count not published, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 61.584239 | 229.7350115555556 | – | 6.018140875 | – | – | sourced | – | source |
| M1 Ultra (GPU count not published, 128 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 57.07094177777779 | 283.7757922222222 | – | 4.787078634222223 | – | – | sourced | – | source |
| M2 (10-core GPU, 8 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 56.46111377777777 | 804.8249559999999 | – | 1.6694184073333336 | – | – | sourced | – | source |
| M2 (10-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 55.565495822222225 | 801.8366849333333 | – | 1.589305650066667 | – | – | sourced | – | source |
| M2 (10-core GPU, 24 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 54.7565285 | 813.7094306666667 | – | 1.5797905915 | – | – | sourced | – | source |
| M5 | llama-3-2-1b-instruct | bf16 | oMLX | 50.2 | 4157.0 | – | – | 4096 | 2026-10-12 | community | – | source |
| M5 | llama-3-2-1b-instruct | bf16 | oMLX | 42.7 | 3651.0 | – | – | 16384 | 2026-10-12 | community | – | source |
| M1 (8-core GPU, 8 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 40.41441422222222 | 607.4231519333333 | – | 2.1677008980666663 | – | – | sourced | – | source |
| M1 (8-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 40.18323162962963 | 585.7259774074074 | – | 2.24642097837037 | – | – | sourced | – | source |
| M1 (7-core GPU, 8 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 38.51773918518518 | 532.9676827037036 | – | 2.4294918625925925 | – | – | sourced | – | source |
| M1 (7-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 37.872518722222225 | 530.8859519722223 | – | 2.4528570579444446 | – | – | sourced | – | source |
| M2 (8-core GPU, 16 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 35.320437777777784 | 523.1656983888889 | – | 3.046989798555555 | – | – | sourced | – | source |
| M2 (8-core GPU, 8 GB) | llama-3-2-1b-instruct | Q4_K - Medium | llamafile | 34.48367238888889 | 523.2109411111111 | – | 2.7225200694166665 | – | – | sourced | – | source |