Mac mini local AI benchmarks

Real numbers from open-weight models running fully local on one machine: an M5 Pro Mac mini with 64GB of unified memory.

Every row is one measurement: model, quant, runtime, the test, and the result, with a link to the post where it was shared. New models get added as I test them.

Each model has its own tab (config) in the viewer, and the all tab puts every model in one table. Every file uses the same columns, so they stack cleanly.

What's in it so far

Flint

Nemotron 3.5 Bolt E4B

NVIDIA Nemotron 3.5 Bolt E4B, 8B total and 4B effective, OptiQ mixed 4/8-bit build on stock mlx-lm. Temp 0 with thinking on. Measured with the Launch Benchmarker Grok Bot.

How it's measured

flint_bench.py times any OpenAI-compatible local server (Ollama, etc.) with only the Python standard library. It reports time to first token, reasoning share, and decode tok/s, as the median of 3 runs.

The Nemotron rows come from the Launch Benchmarker Grok Bot. It runs the model with MLX on the Mac, times decode, prompt processing and time to first token at 512, 4k, 16k and 32k context (median of 3 runs), records MLX's own peak memory number, and runs a fixed small suite of math, code, instruction following and needle-in-a-haystack tests. Paste in any new open-weight model and it does the same run on your own Mac.

Coming next

More launch-day models as they drop, each in its own tab, plus the 11 models from my weekend run on the same Mac.

By @pratikg.

Write-up on GitHub

How I ran this, what broke, and the scripts: Flint on my Mac mini and NVIDIA Nemotron 3.5 Bolt E4B on my Mac mini. It's part of Local AI field notes, where I log every model I run on my Mac mini.