Skip to main content

Blog

Articles, guides, and insights on local artificial intelligence

9 articles found per "token"

Turnbook: local memory for coding agents, without a single model call
Use casesIntermediate

Turnbook: local memory for coding agents, without a single model call

Every conversation stored word for word in a SQLite file, with no model call at all: search runs on local embeddings and retrieval costs 3.1k tokens. 91.8% on LongMemEval, zero tokens to save.

iaLocale Team
November 20, 2026
QRAE: compressing DeepSearch agent context to save tokens and latency
ModelsIntermediate

QRAE: compressing DeepSearch agent context to save tokens and latency

QRAE uses the model's own attention entropy to decide which retrieved chunks stay in the context: tokens -45%, latency -30% and higher success across four benchmarks.

iaLocale Team
November 18, 2026
Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
ModelsIntermediate

Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM

inclusionAI's Ling 3.1 Flash: hybrid MoE 560B total / 25B active, native 1M-token context, FP8 checkpoint, MIT license. What it concretely means for 24GB of VRAM: quantizations, parallelism and API costs.

iaLocale Team
October 19, 2026
Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally
ModelsIntermediate

Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally

Aleph Alpha's Kolibri (78.1B/3.46B, Apache 2.0, FP8) is the European open-weight MoE: reasoning, tool calling and context up to 1M tokens. In FP8 it weighs ~78GB of VRAM: it runs on an RTX 5090 32GB + shared RAM or on unified-memory Macs; for practical adoption the kolibri1 architecture in llama.cpp and Ollama/LM Studio builds are still missing.

iaLocale Team
October 18, 2026
Decision models in local: JEV-35B and the 3B-active System 1
Use casesIntermediate

Decision models in local: JEV-35B and the 3B-active System 1

You don't need a big LLM to decide. JEV-35B (35B, 3B active) in NVFP4 on a 32GB GPU and ej (11.4MB) on CPU show how to do routing, classification and quality control without generating a single token.

iaLocale Team
October 17, 2026
What a token costs: five routes, the math, and the break-even point
ComparisonsIntermediate

What a token costs: five routes, the math, and the break-even point

Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.

iaLocale Team
October 15, 2026
Speculative decoding and MTP: speed for free
FundamentalsIntermediate

Speculative decoding and MTP: speed for free

Speculative decoding makes local AI faster without changing output quality: a small model drafts the tokens, the big model verifies them. Here's how the draft-verify loop works, MTP heads in Qwen models, EAGLE on Mistral Medium 3.5, and how to enable it all in three commands.

iaLocale Team
October 12, 2026
Costs: local vs API, done with real numbers
Use casesIntermediate

Costs: local vs API, done with real numbers

How much does local really cost compared to a cloud API? Verified per-token prices, GPU amortization, electricity and the break-even point, calculated with real numbers.

iaLocale Team
October 6, 2026
Quantization: Get More From the Same Hardware
ModelsIntermediate

Quantization: Get More From the Same Hardware

Quantization reduces the precision of the weights (from FP16 to 4–8 bit) so models fit in less VRAM and generate more tokens per second. The levels, the real file sizes of Qwen2.5 14B, the rule for choosing from Q4_K_M, and the limits (code, reasoning, languages other than English).

iaLocale Team
September 28, 2026