All Benchmarked Models

263 models tested across real hardware. Click any model to see detailed benchmark results.

qwen3.5:9b

qwen35 · 9.7B · Q4_K_M

10 benchmarks avg 76/100 35.6 tok/s

gpt-oss:20b

gptoss · 20.9B · MXFP4

9 benchmarks avg 91/100 64.3 tok/s

qwen/qwen3-30b-a3b-2507

qwen3_moe · 30B · 4bit

9 benchmarks avg 90/100 118.2 tok/s

openai/gpt-oss-20b

gpt_oss · 20B · MXFP4

7 benchmarks avg 86/100 77.8 tok/s

gemma-4-26b-a4b-it-qat

gemma4 · 26B · Q4_K_XL

7 benchmarks avg 90/100 165.2 tok/s

qwen3.5:4b

qwen35 · 4.7B · Q4_K_M

7 benchmarks avg 72/100 32.9 tok/s

qwen3.5:27b

qwen35 · 27.8B · Q4_K_M

6 benchmarks avg 52/100 6.2 tok/s

gemma4:26b

gemma4 · 25.8B · Q4_K_M

5 benchmarks avg 78/100 45.4 tok/s

qwen3:14b

qwen3 · 14.8B · Q4_K_M

5 benchmarks avg 79/100 28.9 tok/s

gemma4:12b

gemma4 · 11.9B · Q4_K_M

5 benchmarks avg 85/100 23.5 tok/s

qwen3:8b

qwen3 · 8.2B · Q4_K_M

5 benchmarks avg 75/100 101.8 tok/s

gemma3:1b

gemma3 · 999.89M · Q4_K_M

5 benchmarks avg 62/100 36.1 tok/s

qwen3-coder:30b

qwen3moe · 30.5B · Q4_K_M

4 benchmarks avg 85/100 81.7 tok/s

qwen3-vl-4b-instruct

qwen3vl · 4B · Q4_K_M

4 benchmarks avg 87/100 71.5 tok/s

qwen3.5-9b

qwen35 · 9B · Q4_K_S

4 benchmarks avg 77/100 23.5 tok/s

gemma4:31b

gemma4 · 31.3B · Q4_K_M

4 benchmarks avg 76/100 13.4 tok/s

qwen3.5:0.8b

qwen35 · 873.44M · Q8_0

4 benchmarks avg 52/100 47.7 tok/s

gemma4:12b-mlx

3 benchmarks avg 85/100 25.7 tok/s

gemma4:e4b-mlx

3 benchmarks avg 84/100 40.1 tok/s

qwen/qwen3.6-35b-a3b

qwen3_5_moe · 35B · 4bit

3 benchmarks avg 87/100 110.9 tok/s

glm-4.7-flash:latest

glm4moelite · 29.9B · Q4_K_M

3 benchmarks avg 84/100 51.1 tok/s

qwen3.6-27b-mtp-pi-reasoning

qwen35 · 27B · Q4_K_M

3 benchmarks avg 85/100 53.2 tok/s

gemma4:e2b

gemma4 · 5.1B · Q4_K_M

3 benchmarks avg 79/100 62.3 tok/s

qwen3.6-35b-a3b

qwen35moe · 35B · Q4_K_S

3 benchmarks avg 81/100 39.8 tok/s

gemma3:4b

gemma3 · 4.3B · Q4_K_M

3 benchmarks avg 67/100 21.5 tok/s

qwen3.5-9b-mlx

qwen3_5 · 9B · 4bit

3 benchmarks avg 65/100 17.1 tok/s

qwen3:0.6b

qwen3 · 751.63M · Q4_K_M

3 benchmarks avg 65/100 105.5 tok/s

mistral:latest

llama · 7.2B · Q4_K_M

3 benchmarks avg 73/100 36.2 tok/s

qwen3.5-35b-a3b-uncensored-hauhaucs-aggressive

qwen35moe · 35B · Q4_K_M

3 benchmarks avg 68/100 62.5 tok/s

qwen2.5:1.5b

qwen2 · 1.5B · Q4_K_M

3 benchmarks avg 67/100 40.2 tok/s

deepseek-r1:1.5b

qwen2 · 1.8B · Q4_K_M

3 benchmarks avg 54/100 46.1 tok/s

qwen3.5-0.8b-mlx

qwen3_5 · 0.8B · 8bit

3 benchmarks avg 56/100 76.4 tok/s

qwen3:4b

qwen3 · 4.0B · Q4_K_M

3 benchmarks avg 36/100 15.7 tok/s

qwen3-30b-a3b-thinking-2507-claude-4.5-sonnet-high-reasoning-distill-mlx

qwen3_moe · 30B · MXFP4

2 benchmarks avg 96/100 80 tok/s

qwen3.6-35b-a3b-claude-4.7-opus-reasoning-distilled-apex-mtp

qwen35moe · 35B

2 benchmarks avg 91/100 42.4 tok/s

qwen3-coder:30b-a3b-q4_K_M

qwen3moe · 30.5B · Q4_K_M

2 benchmarks avg 72/100 102.3 tok/s

qwen3.6:35b-a3b-q8_0

qwen35moe · 36.0B · Q8_0

2 benchmarks avg 86/100 55.8 tok/s

qwen3.6-35b-a3b-claude-4.7-opus-reasoning-distilled-apex

qwen35moe · 35B

2 benchmarks avg 87/100 117.1 tok/s

qwen3-coder:30b-a3b-q8_0

qwen3moe · 30.5B · Q8_0

2 benchmarks avg 84/100 25.5 tok/s

unsloth/gemma-4-26b-a4b-it

gemma4 · 26B · Q3_K_M

2 benchmarks avg 87/100 20.7 tok/s

gemma4:latest

gemma4 · 8.0B · Q4_K_M

2 benchmarks avg 88/100 44.4 tok/s

nemotron-3-nano:latest

nemotron_h_moe · 31.6B · Q4_K_M

2 benchmarks avg 88/100 62.5 tok/s

qwen3.6:35b-mlx

nvfp4

2 benchmarks avg 85/100 129.6 tok/s

qwen3.6-27b-mtp

qwen35 · 27B · Q5_K_M

2 benchmarks avg 88/100 59.4 tok/s

lfm2:24b

lfm2moe · 23.8B · Q4_K_M

2 benchmarks avg 88/100 90.4 tok/s

huihui-qwen3.6-35b-a3b-claude-4.7-opus-abliterated-mtp

qwen35moe · 35B · Q4_K

2 benchmarks avg 88/100 38.8 tok/s

qwopus3.6-27b-coder-compat-mtp

qwen35 · 27B · Q4_K_M

2 benchmarks avg 88/100 61.9 tok/s

qwen3.5:35b-a3b

qwen35moe · 36.0B · Q4_K_M

2 benchmarks avg 74/100 36.2 tok/s

qwopus3.6-35b-a3b-coder-mtp

qwen35moe · 35B · Q4_K_M

2 benchmarks avg 84/100 107.3 tok/s

qwen2.5-7b-instruct

qwen2 · 7B · 4bit

2 benchmarks avg 83/100 23.2 tok/s

qwen3-vl:4b-instruct

qwen3vl · 4.4B · Q4_K_M

2 benchmarks avg 83/100 90.9 tok/s

qwen3-5-4b-q4-k-m

llama · 4B · Q4_K_M

2 benchmarks avg 83/100 62.7 tok/s

qwen3.6:27b

qwen35 · 27.8B · Q4_K_M

2 benchmarks avg 50/100 10.9 tok/s

gemma-3-4b-it-q4-k-m

llama · 4B · Q4_K_M

2 benchmarks avg 80/100 75 tok/s

qwen2.5-coder:7b

qwen2 · 7.6B · Q4_K_M

2 benchmarks avg 74/100 28.3 tok/s

lfm2.5:latest

lfm2moe · 8.5B · Q4_K_M

2 benchmarks avg 76/100 76.3 tok/s

ministral-3:8b

mistral3 · 8.9B · Q4_K_M

2 benchmarks avg 78/100 30.5 tok/s

lfm2-24b-a2b

2 benchmarks avg 48/100 56.2 tok/s

qwen2.5:3b

qwen2 · 3.1B · Q4_K_M

2 benchmarks avg 77/100 34.3 tok/s

phi4:14b

phi3 · 14.7B · Q4_K_M

2 benchmarks avg 76/100 16.9 tok/s

deepseek-r1:7b

qwen2 · 7.6B · Q4_K_M

2 benchmarks avg 72/100 28.9 tok/s

llama3.2:latest

llama · 3.2B · Q4_K_M

2 benchmarks avg 75/100 77.9 tok/s

qwen3.5-27b

qwen3_5 · 27B · 4bit

2 benchmarks avg 51/100 8 tok/s

llama3.1:8b

llama · 8.0B · Q4_K_M

2 benchmarks avg 61/100 14.1 tok/s

liquid/lfm2-24b-a2b

lfm2_moe · 24B · 4bit

2 benchmarks avg 74/100 55.3 tok/s

phi4-mini:latest

phi3 · 3.8B · Q4_K_M

2 benchmarks avg 73/100 27.3 tok/s

llama3.2:3b

llama · 3.2B · Q4_K_M

2 benchmarks avg 74/100 32.6 tok/s

qwen3.5:2b

qwen35 · 2.3B · Q8_0

2 benchmarks avg 43/100 17.5 tok/s

mistral:7b

llama · 7.2B · Q4_K_M

2 benchmarks avg 71/100 22.2 tok/s

qwen3.5-4b-mlx

qwen3_5 · 4B · 4bit

2 benchmarks avg 67/100 11.6 tok/s

qwen3.5-2b-mlx

qwen3_5 · 2B · 8bit

2 benchmarks avg 68/100 40.9 tok/s

qwen2.5-1.5b-instruct

qwen2 · 1.5B · 8bit

2 benchmarks avg 65/100 32.4 tok/s

google/gemma-4-e4b

gemma4 · Q4_K_M

2 benchmarks avg 65/100 26.9 tok/s

smollm2-1.7b-instruct

llama · 1.7B · bf16

2 benchmarks avg 57/100 17.5 tok/s

starling-lm:7b

llama · 7B · Q4_0

2 benchmarks avg 64/100 23.7 tok/s

microsoft/phi-4-mini-reasoning

phi3 · 3.8B · 4bit

2 benchmarks avg 63/100 65.3 tok/s

llama3.2:1b

llama · 1.2B · Q8_0

2 benchmarks avg 63/100 105.2 tok/s

smollm2-360m-instruct

llama · 360M · bf16

2 benchmarks avg 50/100 70.5 tok/s

starcoder2:3b

starcoder2 · 3B · Q4_0

2 benchmarks avg 37/100 40.5 tok/s

deepseek-r1:latest

qwen2 · 7.6B · Q4_K_M

2 benchmarks avg 23/100 9.7 tok/s

nemotron3:33b

nemotron_h_omni · 33.0B · Q4_K_M

1 benchmarks avg 96/100 62.7 tok/s

gpt-oss-20b

1 benchmarks avg 95/100 118.2 tok/s

nemotron-3-nano

1 benchmarks avg 93/100 93.3 tok/s

qwen/qwen3-vl-30b

qwen3vlmoe · 30B · Q4_K_M

1 benchmarks avg 92/100 24.8 tok/s

gemma-4-31b-it-qat

gemma4 · 31B · Q4_K_XL

1 benchmarks avg 91/100 40.2 tok/s

qwen/qwen3-8b

qwen3 · 8B · 4bit

1 benchmarks avg 89/100 33.4 tok/s

gemma4:12b-it-qat

gemma4 · 11.9B · Q4_0

1 benchmarks avg 88/100 30 tok/s

qwen3.6:35b-a3b

qwen35moe · 36.0B · Q4_K_M

1 benchmarks avg 88/100 43.3 tok/s

gpt-oss-orchestrator:latest

gptoss · 20.9B · MXFP4

1 benchmarks avg 88/100 24.6 tok/s

qwen/qwen3-coder-30b

qwen3moe · 30B · Q4_K_M

1 benchmarks avg 88/100 95.4 tok/s

gpt-oss-safeguard-20b-mlx

gpt_oss · 20B · MXFP4

1 benchmarks avg 87/100 41.4 tok/s

qwen3:30b

qwen3moe · 30.5B · Q4_K_M

1 benchmarks avg 86/100 204.8 tok/s

unsloth/gemma-4-26b-a4b-it-qat

gemma4 · 26B · Q4_K_XL

1 benchmarks avg 85/100 11.9 tok/s

gemma4:26b-nvfp4

nvfp4

1 benchmarks avg 85/100 65.7 tok/s

qwen2.5:7b

qwen2 · 7.6B · Q4_K_M

1 benchmarks avg 84/100 22.2 tok/s

gemma4:e4b

gemma4 · 8.0B · Q4_K_M

1 benchmarks avg 84/100 53 tok/s

nvidia/nemotron-3-nano

nemotron_h · 30B · 4bit

1 benchmarks avg 84/100 47.9 tok/s

gptoss · 20.9B · MXFP4

gptoss · 20.9 · MXFP4

1 benchmarks avg 84/100 17.5 tok/s

qwen3:30b-a3b

qwen3moe · 30.5B · Q4_K_M

1 benchmarks avg 84/100 92.7 tok/s

qwen2.5:14b

qwen2 · 14.8B · Q4_K_M

1 benchmarks avg 83/100 11.2 tok/s

zai-org/glm-4.7-flash

deepseek2 · 30B · Q4_K_M

1 benchmarks avg 83/100 102.9 tok/s

ministral-3-14b-instruct-2512

mistral3 · 14B · Q5_K_M

1 benchmarks avg 83/100 83.7 tok/s

gemma-4-26b-a4b-it@iq3_xxs

gemma4 · 26B · IQ3_XXS

1 benchmarks avg 83/100 10.3 tok/s

unsloth/gemma-4-26b-a4b-it@q4_k_m

gemma4 · 26B · Q4_K_M

1 benchmarks avg 83/100 9.4 tok/s

qwen3-coder-next:latest

qwen3next · 79.7B · Q4_K_M

1 benchmarks avg 82/100 34.5 tok/s

unsloth-phi-4

llama · 4bit

1 benchmarks avg 82/100 27.4 tok/s

deepseek-coder-v2:16b

deepseek2 · 15.7B · Q4_0

1 benchmarks avg 82/100 117.2 tok/s

google/gemma-4-26b-a4b-it-qat

gemma4 · 26B · Q4_0

1 benchmarks avg 82/100 12 tok/s

gemma3n:latest

gemma3n · 6.9B · Q4_K_M

1 benchmarks avg 81/100 108.7 tok/s

llama-3.1-8b-instruct

llama · 8B · Q8_0

1 benchmarks avg 81/100 35.6 tok/s

gemma-4-e2b-it-qat

gemma4 · Q4_K_XL

1 benchmarks avg 81/100 22.2 tok/s

lmstudio-community/gemma-4-26b-a4b-it

gemma4 · 26B · Q4_K_M

1 benchmarks avg 81/100 9.9 tok/s

glm-4.7-flash:q4_K_M

glm4moelite · 29.9B · Q4_K_M

1 benchmarks avg 81/100 35.1 tok/s

qwen3-vl:30b

qwen3vlmoe · 31.1B · Q4_K_M

1 benchmarks avg 80/100 208.9 tok/s

gemma3:12b

gemma3 · 12.2B · Q4_K_M

1 benchmarks avg 80/100 12.7 tok/s

qwen3.5:122b-a10b

qwen35moe · 125.1B · Q4_K_M

1 benchmarks avg 80/100 19.1 tok/s

mlx-community/gemma-3-4b-it-qat-4bit

1 benchmarks avg 80/100 43.2 tok/s

qwen3.5-35b-a3b

qwen35moe · 35B · IQ3_XXS

1 benchmarks avg 80/100 10.6 tok/s

qwen2.5-coder-7b-instruct-mlx

qwen2 · 7B · 4bit

1 benchmarks avg 79/100 22.5 tok/s

qwen3:32b

qwen3 · 32.8B · Q4_K_M

1 benchmarks avg 79/100 9.7 tok/s

slim-rpcache-ornith-1.0-9b

qwen35 · 9B · BF16

1 benchmarks avg 79/100 52.3 tok/s

unsloth/gemma-4-e4b-it@q4_k_m

gemma4 · Q4_K_M

1 benchmarks avg 79/100 10.8 tok/s

unsloth/gemma-4-e4b-it-qat

gemma4 · Q4_K_XL

1 benchmarks avg 79/100 12.1 tok/s

gemma-4-e2b-it@q5_k_xl

gemma4 · Q5_K_XL

1 benchmarks avg 79/100 17.2 tok/s

qwen2.5-coder:32b

qwen2 · 32.8B · Q4_K_M

1 benchmarks avg 78/100 9.5 tok/s

lfm2:24b-a2b

lfm2moe · 23.8B · Q4_K_M

1 benchmarks avg 78/100 11.5 tok/s

google/gemma-4-e4b-it-qat

gemma4 · Q4_0

1 benchmarks avg 78/100 11.4 tok/s

lmstudio-community/meta-llama-3.1-8b-instruct

llama · 8B · Q4_K_M

1 benchmarks avg 78/100 46 tok/s

yi:6b

llama · 6B · Q4_0

1 benchmarks avg 77/100 27.6 tok/s

gemma2:2b

gemma2 · 2.6B · Q4_0

1 benchmarks avg 77/100 55.1 tok/s

internlm2:7b

internlm2 · 7.7B · Q4_0

1 benchmarks avg 77/100 22.7 tok/s

gemma-4-e4b-it@q5_k_xl

gemma4 · Q5_K_XL

1 benchmarks avg 77/100 9.3 tok/s

lmstudio-community/gemma-4-e4b-it-qat

gemma4 · Q4_0

1 benchmarks avg 77/100 11.4 tok/s

lmstudio-community/gemma-4-e2b-it

gemma4 · Q8_0

1 benchmarks avg 77/100 13.3 tok/s

mlx-community/meta-llama-3.1-8b-instruct

llama · 8B · 4bit

1 benchmarks avg 77/100 54.3 tok/s

qwen3.5:35b

qwen35moe · 36.0B · Q4_K_M

1 benchmarks avg 77/100 16 tok/s

qwen3.5-35b-a3b-mtp

qwen35moe · 35B · IQ3_XXS

1 benchmarks avg 77/100 7.8 tok/s

qwen3.6-35b-a3b-mtp

qwen35moe · 35B · IQ3_XXS

1 benchmarks avg 77/100 9.2 tok/s

cogito:8b

llama · 8.0B · Q4_K_M

1 benchmarks avg 77/100 21 tok/s

glm-4.7-flash:bf16

deepseek2 · 29.9B · F16

1 benchmarks avg 76/100 25.8 tok/s

gemma2:9b

gemma2 · 9.2B · Q4_0

1 benchmarks avg 76/100 17.5 tok/s

qwen3-coder:latest

qwen3moe · 30.5B · Q4_K_M

1 benchmarks avg 76/100 7.6 tok/s

glm4:9b

chatglm · 9.4B · Q4_0

1 benchmarks avg 76/100 19 tok/s

google/gemma-3-4b

gemma3 · 4B · 4bit

1 benchmarks avg 76/100 39.4 tok/s

lmstudio-community/gemma-4-e4b-it

gemma4 · Q4_K_M

1 benchmarks avg 76/100 10.8 tok/s

qwen3.6:35b-a3b-coding-nvfp4

nvfp4

1 benchmarks avg 76/100 72.7 tok/s

gemma-3-27b-it-qat

gemma3 · 27B · 4bit

1 benchmarks avg 76/100 8.9 tok/s

exaone-3.5-2.4b-instruct-mlx

exaone · 2.4B · 8bit

1 benchmarks avg 75/100 37 tok/s

mistral-small:24b

llama · 23.6B · Q4_K_M

1 benchmarks avg 75/100 14.3 tok/s

qwen2.5:72b

qwen2 · 72.7B · Q4_K_M

1 benchmarks avg 75/100 4.4 tok/s

lmstudio-community/gemma-4-12b-it-qat

gemma4 · 12B · Q4_0

1 benchmarks avg 75/100 5.8 tok/s

unsloth/gemma-4-12b-it-qat

gemma4 · 12B · Q4_K_XL

1 benchmarks avg 75/100 6 tok/s

unsloth/gemma-4-e2b-it@q8_0

gemma4 · Q8_0

1 benchmarks avg 75/100 13 tok/s

granite3.1-dense:8b

granite · 8.2B · Q4_K_M

1 benchmarks avg 74/100 18.8 tok/s

minimax-m2.7:cloud

minimax

1 benchmarks avg 74/100 0 tok/s

mlx-community/Llama-3.2-3B-Instruct-4bit

1 benchmarks avg 74/100 50.6 tok/s

google/gemma-4-12b-it-qat

gemma4 · 12B · Q4_0

1 benchmarks avg 74/100 5 tok/s

hermes3:8b

llama · 8.0B · Q4_0

1 benchmarks avg 74/100 22.3 tok/s

dolphin3:8b

llama · 8.0B · Q4_K_M

1 benchmarks avg 74/100 21 tok/s

mistralai/magistral-small-2509

mistral3 · 24B · 4bit

1 benchmarks avg 74/100 7.9 tok/s

qwen2.5-coder:3b

qwen2 · 3.1B · Q4_K_M

1 benchmarks avg 73/100 21.3 tok/s

qwen2.5-coder-3b-instruct-mlx

qwen2 · 3B · 4bit

1 benchmarks avg 73/100 52.3 tok/s

llama3.3:70b

llama · 70.6B · Q4_K_M

1 benchmarks avg 73/100 4.6 tok/s

mlx-community/Yi-1.5-6B-Chat-4bit

1 benchmarks avg 73/100 29.1 tok/s

unsloth/qwen3.5-9b@q4_k_m

qwen35 · 9B · Q4_K_M

1 benchmarks avg 73/100 6.5 tok/s

qwen3.5-9b@q5_k_xl

qwen35 · 9B · Q5_K_XL

1 benchmarks avg 73/100 5.6 tok/s

qwen3:1.7b

qwen3 · 2.0B · Q4_K_M

1 benchmarks avg 72/100 120.7 tok/s

qwen3-vl:8b

qwen3vl · 8.8B · Q4_K_M

1 benchmarks avg 72/100 68.2 tok/s

mlx-community/gemma-3-1b-it-8bit

1 benchmarks avg 72/100 86.3 tok/s

aya-expanse:8b

command-r · 8.0B · Q4_K_M

1 benchmarks avg 72/100 19.7 tok/s

codegemma:7b

gemma · 9B · Q4_0

1 benchmarks avg 71/100 18.7 tok/s

mistral-nemo:12b

llama · 12.2B · Q4_0

1 benchmarks avg 71/100 14.6 tok/s

ministral-3:3b

mistral3 · 3.8B · Q4_K_M

1 benchmarks avg 71/100 42.2 tok/s

llama3:latest

llama · 8.0B · Q4_0

1 benchmarks avg 71/100 22.3 tok/s

google/gemma-3-27b

gemma3 · 27B · 4bit

1 benchmarks avg 71/100 5.9 tok/s

qwen2.5-coder-1.5b-instruct-mlx

qwen2 · 1.5B · 8bit

1 benchmarks avg 70/100 58.4 tok/s

qwen3.6:35b

qwen35moe · 36.0B · Q4_K_M

1 benchmarks avg 70/100 30.6 tok/s

yi-coder:9b

llama · 8.8B · Q4_0

1 benchmarks avg 70/100 19.4 tok/s

deepseek-v2:16b

deepseek2 · 15.7B · Q4_0

1 benchmarks avg 69/100 56.6 tok/s

mistralai/devstral-small-2-2512

mistral3 · 24B · 4bit

1 benchmarks avg 69/100 5.7 tok/s

lfm2.5-1.2b-instruct-mlx

lfm2 · 1.2B · 8bit

1 benchmarks avg 68/100 77.3 tok/s

granite3.1-dense:2b

granite · 2.5B · Q4_K_M

1 benchmarks avg 68/100 54.6 tok/s

nous-hermes2:latest

llama · 11B · Q4_0

1 benchmarks avg 68/100 15.9 tok/s

cogito:3b

llama · 3.6B · Q4_K_M

1 benchmarks avg 67/100 45.6 tok/s

codestral:22b

llama · 22.2B · Q4_0

1 benchmarks avg 67/100 17.2 tok/s

neural-chat:7b

llama · 7B · Q4_0

1 benchmarks avg 67/100 22.5 tok/s

aya:8b

command-r · 8.0B · F16

1 benchmarks avg 66/100 20 tok/s

granite-3.3-2b-instruct

granite · 2B · bf16

1 benchmarks avg 66/100 18.3 tok/s

granite3.1-moe:3b

granitemoe · 3.3B · Q4_K_M

1 benchmarks avg 66/100 27.2 tok/s

qwen3-4b-qwen3.6-plus-reasoning-distilled

qwen3 · 4B · Q4_1

1 benchmarks avg 65/100 99.6 tok/s

glm-4.6v-flash

glm4 · Q4_K_S

1 benchmarks avg 65/100 49.9 tok/s

richardyoung/qwythos-9b-abliterated:Q8_0

qwen35 · 9.0B · Q8_0

1 benchmarks avg 65/100 25.3 tok/s

smollm2:1.7b

llama · 1.7B · Q8_0

1 benchmarks avg 64/100 51.8 tok/s

phi3:3.8b

phi3 · 3.8B · Q4_0

1 benchmarks avg 64/100 42 tok/s

falcon-h1-1.5b-instruct

falcon-h1 · 1.5B · Q4_K_M

1 benchmarks avg 63/100 6.6 tok/s

solar:10.7b

llama · 11B · Q4_0

1 benchmarks avg 63/100 15.8 tok/s

phi3:14b

phi3 · 14.0B · Q4_0

1 benchmarks avg 63/100 12.7 tok/s

codellama:7b

llama · 7B · Q4_0

1 benchmarks avg 63/100 23.6 tok/s

qwen3-vl-4b-instruct-q4_k_m.gguf

1 benchmarks avg 62/100 67.8 tok/s

qwen3-vl-4b-instruct-q4_k_m

1 benchmarks avg 62/100 67.2 tok/s

qwen3-vl:2b

qwen3vl · 2.1B · Q4_K_M

1 benchmarks avg 62/100 127.4 tok/s

dolphin-phi:2.7b

phi2 · 3B · Q4_0

1 benchmarks avg 61/100 56.1 tok/s

qwen2.5-coder-1.5b-instruct

qwen2 · 1.5B · Q4_K_M

1 benchmarks avg 61/100 7.3 tok/s

lfm2.5-thinking:1.2b

lfm2 · 1.2B · Q4_K_M

1 benchmarks avg 60/100 227 tok/s

text-embedding-nomic-embed-text-v1.5

nomic-bert · Q4_K_M

1 benchmarks avg 59/100 45.3 tok/s

vicuna:7b

llama · 7B · Q4_0

1 benchmarks avg 59/100 23.4 tok/s

vicuna:13b

llama · 13B · Q4_0

1 benchmarks avg 59/100 13.4 tok/s

wizardlm2:7b

llama · 7B · Q4_0

1 benchmarks avg 59/100 23.2 tok/s

gemma-3-1b-it

gemma3 · 1B · Q4_K_M

1 benchmarks avg 59/100 8.7 tok/s

phi:2.7b

phi2 · 3B · Q4_0

1 benchmarks avg 59/100 56.6 tok/s

gemma-2-2b-it

gemma2 · 2B · Q4_K_M

1 benchmarks avg 58/100 3.6 tok/s

falcon-h1-0.5b-instruct

falcon-h1 · 0.5B · Q4_K_M

1 benchmarks avg 57/100 16.3 tok/s

qwen2.5:0.5b

qwen2 · 494.03M · Q4_K_M

1 benchmarks avg 57/100 165 tok/s

mlx-community/Llama-3.2-1B-Instruct-4bit

1 benchmarks avg 57/100 126.4 tok/s

deepseek-coder:6.7b

llama · 7B · Q4_0

1 benchmarks avg 57/100 24.1 tok/s

llama2:13b

llama · 13B · Q4_0

1 benchmarks avg 56/100 13.5 tok/s

orca-mini:7b

llama · 7B · Q4_0

1 benchmarks avg 55/100 24.4 tok/s

phi-3.5-mini-instruct

phi3 · 4bit

1 benchmarks avg 55/100 43.4 tok/s

qwen2.5-coder-0.5b-instruct

qwen2 · 0.5B · Q4_K_M

1 benchmarks avg 55/100 13.5 tok/s

llama2:7b

llama · 7B · Q4_0

1 benchmarks avg 55/100 25.1 tok/s

internlm2_5-1_8b-chat

internlm2 · Q4_K_M

1 benchmarks avg 55/100 6.6 tok/s

falcon3-1b-instruct

llama · 1B · 3bit

1 benchmarks avg 54/100 121.8 tok/s

mlx-community/Nanbeige4.1-3B-8bit

1 benchmarks avg 53/100 25.4 tok/s

qwen2.5-0.5b-instruct

qwen2 · 0.5B · Q4_K_M

1 benchmarks avg 53/100 13.2 tok/s

qwen2.5-0.5b-instruct-mlx

qwen2 · 0.5B · 4bit

1 benchmarks avg 52/100 257.5 tok/s

smollm2:360m

llama · 361.82M · F16

1 benchmarks avg 52/100 117.9 tok/s

gemma3:270m

gemma3 · 268.10M · Q8_0

1 benchmarks avg 50/100 239.2 tok/s

granite-3.1-1b-a400m-instruct

granitemoe · 1B · Q4_K_M

1 benchmarks avg 50/100 19.6 tok/s

llama-3.2-1b-instruct

llama · 1B · Q4_K_M

1 benchmarks avg 50/100 9.9 tok/s

mlx-community/stablelm-2-zephyr-1_6b-4bit

1 benchmarks avg 50/100 93.6 tok/s

qwen3.5-0.8b

qwen35 · 0.8B · Q4_K_M

1 benchmarks avg 48/100 6.2 tok/s

orca-mini:3b

llama · 3B · Q4_0

1 benchmarks avg 47/100 30.6 tok/s

deepseek-r1:70b

llama · 70.6B · Q4_K_M

1 benchmarks avg 47/100 4.7 tok/s

stablelm2:1.6b

stablelm · 2B · Q4_0

1 benchmarks avg 45/100 93.8 tok/s

qwen2.5-math-1.5b-instruct

qwen2 · 1.5B · 4bit

1 benchmarks avg 45/100 91.7 tok/s

mlx-community/quantized-gemma-2b-it

1 benchmarks avg 45/100 44.4 tok/s

yi-coder-1.5b-chat

llama · 1.5B · Q4_K_M

1 benchmarks avg 45/100 7.7 tok/s

opencoder-1.5b-instruct

llama · 1.5B · Q4_K_M

1 benchmarks avg 44/100 4.8 tok/s

amd-olmo-1b-sft

olmo · 1B · Q4_K_M

1 benchmarks avg 44/100 9.2 tok/s

tinyllama:1.1b

llama · 1B · Q4_0

1 benchmarks avg 43/100 121 tok/s

falcon-h1-tiny-90m-instruct

falcon-h1 · 90M · Q4_K_M

1 benchmarks avg 43/100 67.8 tok/s

tinyllama

1 benchmarks avg 43/100 133.4 tok/s

gemma-3-270m-it-qat-mlx

gemma3_text · 270M · 4bit

1 benchmarks avg 43/100 344.2 tok/s

smollm2:135m

llama · 134.52M · F16

1 benchmarks avg 41/100 241 tok/s

qwen3-0.6b

qwen3 · 0.6B · Q4_K_M

1 benchmarks avg 40/100 14.4 tok/s

phi-3-mini-128k-instruct

phi3 · 4bit

1 benchmarks avg 38/100 44 tok/s

smollm2-135m-instruct

llama · 135M · Q4_K_M

1 benchmarks avg 38/100 39.3 tok/s

qwen36-distill-apex

1 benchmarks avg 36/100 55.2 tok/s

deepseek-r1-distill-qwen-14b-mlx

qwen2 · 14B · 5bit

1 benchmarks avg 36/100 10.3 tok/s

stablelm-2-zephyr-1.6b

stablelm · 1.6B · Q4_K_M

1 benchmarks avg 35/100 6.5 tok/s

starcoder2:7b

starcoder2 · 7B · Q4_0

1 benchmarks avg 35/100 23.8 tok/s

qwen3.5-27b-claude-4.6-opus-distilled-mlx

qwen3_5 · 27B · 4bit

1 benchmarks avg 34/100 6.1 tok/s

microllm-250m:latest

llama · 256.74M · F16

1 benchmarks avg 34/100 291.6 tok/s

qwen3.5:latest

qwen35 · 9.7B · Q4_K_M

1 benchmarks avg 34/100 13.3 tok/s

tinyllama-1.1b-chat-v1.0

llama · 1.1B · Q3_K_M

1 benchmarks avg 33/100 9.9 tok/s

microllm-10m:latest

llama · 10.00M · F16

1 benchmarks avg 33/100 1544.8 tok/s

lmstudio-community/Phi-4-reasoning-plus-MLX-4bit

1 benchmarks avg 33/100 11.5 tok/s

bloomz-560m

bloom · 560M · Q4_K_M

1 benchmarks avg 30/100 12.2 tok/s

deepseek-coder-1.3b-instruct

llama · 1.3B · Q8_0

1 benchmarks avg 30/100 7.1 tok/s

phi-1_5

phi2 · Q8_0

1 benchmarks avg 28/100 6.9 tok/s

microllm-50m:latest

llama · 49.31M · F16

1 benchmarks avg 28/100 1000000 tok/s

deepseek-r1:14b

qwen2 · 14.8B · Q4_K_M

1 benchmarks avg 22/100 11.4 tok/s

qwen3-1.7b

qwen3 · 1.7B · Q4_K_M

1 benchmarks avg 19/100 5.9 tok/s