Apple integrates MLX 0.20 with native support for quantized models
On-device & edgeRelated articles
GGUF: The Format That Keeps Models on Your Hardware
We open the box: what's really inside a .gguf file, why it's a single file, how to read the filename, and how to pick the size for your VRAM, with the real sizes of an 8B model.
Quantization: Get More From the Same Hardware
What quantization is, the Q4/Q5/Q8 levels, and how to pick the right one for your VRAM, with a concrete example on Qwen2.5 14B.
Mac Studio M5 Ultra: practical 512GB guide - models that fit and GGUF setup
The 512GB Mac Studio holds a 560B MoE at 4 bit with a large context. This guide covers the 4-bit rule, GGUF setup in three paths, the real bottleneck (the KV cache), and the Mac-vs-API cost comparison with the numbers on the table.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.