Local AI on Mac: the complete Apple Silicon guide

The Mac has become the simplest machine to run local AI. No graphics card to choose, no driver to configure: you just install a tool, download a model, and ask your first question. This page covers it all: how much RAM you need, which model to choose based on your chip, and what to expect in terms of speed.

Why Apple Silicon excels at local AI

Two characteristics make the difference.

Unified memory. On a regular PC, the graphics card has its own video memory (VRAM), usually 8 to 24 GB, separate from the system RAM. On an M1, M2, M3, or M4 Mac, the processor and the graphics chip share the same memory pool. The entire model can reside there without copying data from one place to another. Concretely: a 32 GB Mac offers more room for the model than a 16 GB graphics card, and a 64 GB Mac surpasses the 48 GB of an RTX 4090.

Compute cores. Apple's integrated GPUs (up to 40 cores on the Max chips) are designed for matrix computation, which is exactly what a language model does during inference.

How much RAM you need

Mac RAM	Comfortable model	Use
8 GB	3B (Qwen 2.5 3B, Gemma 2B)	Testing, short summaries
16 GB	7B–8B in Q4 (Llama 3.1 8B, Qwen 2.5 7B)	Chat, writing, light code
24–32 GB	12B–14B (Mistral Small, Qwen 2.5 14B)	Comfortable everyday use
48 GB and +	27B–32B (Qwen 2.5 32B)	Serious work, long documents

Rule of thumb: an 8B model in Q4_K_M quantization weighs about 4.7 GB of RAM. Leave 4 to 8 GB of headroom for the system and applications.

Installing Ollama on Mac

In the terminal:

curl -fsSL https://ollama.com/install.sh | sh

or via Homebrew:

brew install ollama

Then download a model:

ollama pull llama3.1:8b
ollama run llama3.1:8b "Bonjour, présente-toi en français"

That's it. Ollama exposes a local API on http://localhost:11434 in an OpenAI-compatible format: tools like Open WebUI, LM Studio, or Dify connect to it without modification.

Which model for your machine

• M1 / M2 8 GB — qwen2.5:3b or gemma2:2b: solid in French and Italian, ideal for testing.
• M1 Pro / M2 16 GB — llama3.1:8b or qwen2.5:7b: the best quality/simplicity ratio for chat and writing.
• M2 Pro / Max 32 GB — qwen2.5:14b or mistral-small: noticeably more comfortable at document comprehension.
• M3 / M4 Max 48–128 GB — qwen2.5:32b and beyond.

For French and Italian, the Qwen, Llama, and Mistral families are the most even. Test two models of the same size on your own documents before deciding.

Realistic performance

Be honest with your expectations. An M1 Pro gets about 8 to 12 tokens per second on a quantized 8B; an M3 Max can exceed 50. That's very different from an RTX 4090 (100 tokens/s and up), but for reading it's more than enough: a human reads at about 3 tokens per second. The weak point is first-response latency: with long documents (RAG), the higher the memory bandwidth, the better.

Best practices

• Keep the system up to date: Ollama regularly improves the MLX engine and performance.
• Don't overload the RAM: a model that's too big and "swaps" to disk becomes unusable.
• Prefer Q4_K_M quantization for everyday use; move to Q5 or Q6 only if you have the headroom.

In summary

A 32 GB Mac is today the simplest entry point into local AI: installation in three commands, quality models in French and Italian, and everything stays on your machine. Start with ollama pull llama3.1:8b and judge the difference on your own documents.

Article generated with AI