Blog
Articles, guides, and insights on local artificial intelligence
10 articles found per "moe"

Qwen3.8-2.4T-A95B: the math of running 2.4T open-weight parameters, 95B active
The first Qwen-Max-class open-weight model (2.4T parameters, 95B active) outgrows any consumer node: the math on weights, multi-node hardware, 36-month amortization, and power costs, and when running it locally makes sense.

Mistral Large 4 in open weights: a 1T MoE (52B active) for on-premise inference
Mistral released Large 4 in open weights (1.03T total, 52B active, 1M context, open-weight license): an article teaching how to read MoE specs and do the RAM, quantization and speed math for on-premise inference.

Nemotron 3.5 Bolt: NVIDIA's 74B MoE with ~9B active — what changes on your desktop
74B parameters, 9B active: NVIDIA's MoE for local agents fits 24GB of VRAM via IQ2 quantization. Real VRAM, declared benchmarks and honest limits.

MiniCPM-V 4.7: the 35B MoE that runs on a smartphone with 3 GB of RAM
MiniCPM-V 4.7 (35B total, ~3B active, released October 12, 2026 by OpenBMB and ModelBest) is the first open MoE in the series, vision-language: multiple images, video, OCR. 13+ tok/s with 3 GB of RAM on an iPhone 17 Pro Max, 17+ on Android. Official quantizations BF16, FP8, INT4, AWQ, GPTQ, GGUF; engines Ollama, llama.cpp, vLLM, SGLang. The three techniques: Lookahead Expert Scheduling, fine per-level quantization, DSpark speculative sampling.

Ling 3.1 Flash: 560B open-weight with MIT license and 1M context — what changes if you have 24GB of VRAM
inclusionAI's Ling 3.1 Flash: hybrid MoE 560B total / 25B active, native 1M-token context, FP8 checkpoint, MIT license. What it concretely means for 24GB of VRAM: quantizations, parallelism and API costs.

Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally
Aleph Alpha's Kolibri (78.1B/3.46B, Apache 2.0, FP8) is the European open-weight MoE: reasoning, tool calling and context up to 1M tokens. In FP8 it weighs ~78GB of VRAM: it runs on an RTX 5090 32GB + shared RAM or on unified-memory Macs; for practical adoption the kolibri1 architecture in llama.cpp and Ollama/LM Studio builds are still missing.

gpt-oss: OpenAI's open reasoning model
21B total parameters but only 3.6B active, native MXFP4 weights and 128K of context: OpenAI's first open reasoning model fits in 16GB. What that means in practice.

How to Read a Model's Spec Sheet: Family, Size, Context
A guide to reading a model's specs: family, total and active size, context window, quantization, benchmarks and license, with a worked example on DeepSeek V4.1 Flash.

Qwen: the open model with the best multilingual support
Qwen is the open family with the best support for Italian and French: generations, sizes, MoE, when to use it, and a quick comparison with Llama.

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware
Llama 3 and 4 in a table: 3B/8B/70B and Scout MoE 109B, RAM required per model, the Community license and the choice by budget.

