Choosing hardware and model

How to pick the right hardware and the model suited to your use: VRAM, quantization, GPU or CPU and response speed.

15 articles in this path

1
1.FundamentalsIntermediate
How to Choose Your First Local AI Model
The three criteria for picking your first local model: use case, memory (with the exact formula) and context, with the recommended models for 8, 16 and 24 GB.
June 9, 2026 · 6 min read
2
2.HardwareIntermediate
Hardware for Local AI: What You Need in 2026
The memory rule (weights + KV cache), real October 2026 prices for GPUs, Apple Silicon and CPU, the 36-month cost math and the choice by budget.
June 10, 2026 · 7 min read
3
3.HardwareIntermediate
GPU or CPU: when the CPU is enough
How prefill and decode work, what matters for speed (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure.
September 21, 2026 · 7 min read
4
4.GuidesIntermediate
Local AI on Mac: the complete Apple Silicon guide
Why Apple Silicon (unified memory, GPU cores) is the simplest machine for local AI: the RAM you need, installing Ollama, recommended models per chip generation, and realistic performance expectations.
September 10, 2026 · 4 min read
5
5.GuidesIntermediate
Running LLM Models Locally: From First Install to First Working Model
Learn how to install and use large language models on your own computer, without relying on cloud services.
June 10, 2026 · 7 min read
6
6.GuidesAdvanced
vLLM: serving models fast
vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the key concepts for choosing when to use it instead of Ollama.
September 20, 2026 · 7 min read
7
7.HardwareIntermediate
Unified memory: from Strix Halo to Mac Studio M5 Ultra — what fits and at what speed
128GB or 512GB of memory shared between CPU and GPU: three real platforms, the bandwidth that determines speed, and the math on what fits and for whom it makes sense.
October 10, 2026 · 8 min read
8
8.ModelsIntermediate
gpt-oss: OpenAI's open reasoning model
21B total parameters but only 3.6B active, native MXFP4 weights and 128K of context: OpenAI's first open reasoning model fits in 16GB. What that means in practice.
October 11, 2026 · 5 min read
9
9.HardwareIntermediate
The 16GB Blackwell cards: 5060 Ti, 5070, 5070 Ti
The 16GB VRAM tier is the center of gravity of local AI in 2026. It's enough memory to run the best open dense models in Q4 formats — the 27B class, the new balance point of the open catalog — without giving up too much context, and a list price that (depending on the month) stays within a personal budget. The Blackwell generation offers three cards exactly in the middle of this tier: the RTX 5060 Ti, 5070 and 5070 Ti. Same generation, same GDDR7, three very different experiences. Here's how they compare: what each one really brings you, and where your budget should go.
October 13, 2026 · 6 min read
10
10.ComparisonsIntermediate
What a token costs: five routes, the math, and the break-even point
Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.
October 15, 2026 · 9 min read
11
11.HardwareIntermediate
Mac Studio M5 Ultra: 512GB of unified memory on the desktop
Four dies, 512GB of unified memory at 1.2 TB/s: the Mac Studio M5 Ultra runs 100-560B MoE models locally. The speed math, the cost comparison with the DGX Station, and when the Mac is the right choice.
October 16, 2026 · 5 min read
12
12.ModelsIntermediate
GGUF: The Format That Keeps Models on Your Hardware
GGUF is the single-file format that ships models for local inference: a header with a signature and version, key-value metadata, a tensor index, and weights aligned for fast mmap loading. Quantization types with real sizes (8B from 14.96 GiB at F16 to 4.58 GiB at Q4_K_M), the filename convention, and the rule for picking a size against your VRAM or RAM.
September 30, 2026 · 7 min read
13
13.ModelsIntermediate
Quantization: Get More From the Same Hardware
Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).
September 28, 2026 · 5 min read
14
14.ModelsIntermediate
When a Big Model Isn't Needed: Choosing by Size
The method for choosing model size by hardware: the three limits (memory, speed, perceived latency), the VRAM table for Q4 quantization, the hardware classes, and the cases where the small model wins.
September 27, 2026 · 6 min read
15
15.ModelsIntermediate
How to Read a Model's Spec Sheet: Family, Size, Context
A guide to reading a model's specs: family, total and active size, context window, quantization, benchmarks and license, with a worked example on DeepSeek V4.1 Flash.
September 26, 2026 · 8 min read

Choosing hardware and model

How to Choose Your First Local AI Model

Hardware for Local AI: What You Need in 2026

GPU or CPU: when the CPU is enough

Local AI on Mac: the complete Apple Silicon guide

Running LLM Models Locally: From First Install to First Working Model

vLLM: serving models fast

Unified memory: from Strix Halo to Mac Studio M5 Ultra — what fits and at what speed

gpt-oss: OpenAI's open reasoning model

The 16GB Blackwell cards: 5060 Ti, 5070, 5070 Ti

What a token costs: five routes, the math, and the break-even point

Mac Studio M5 Ultra: 512GB of unified memory on the desktop

GGUF: The Format That Keeps Models on Your Hardware

Quantization: Get More From the Same Hardware

When a Big Model Isn't Needed: Choosing by Size

How to Read a Model's Spec Sheet: Family, Size, Context