Ollama v0.33.1: Qwen3.8 Flash Next support and structured output on MLX
On-device & edgeRelated articles
vLLM: serving models fast
vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the concepts (throughput, quantization, KV cache) for choosing when to use it instead of Ollama.
GPU or CPU: when the CPU is enough
Do you really need a GPU for local AI? For many use cases, no: how prefill and decode work, what matters (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure on your own hardware.
Running LLM Models Locally: From First Install to First Working Model
From zero to a first answering model: what you actually need, Ollama step by step, how to pick a model for your memory, expected speeds per hardware, and the typical problems fixed.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.