Aleph Alpha's Kolibri: the 78B MoE that runs (almost) locally

Kolibri is Aleph Alpha's open-weight model: 78 billion parameters, of which only 3.46B are activated per token. It is a Mixture-of-Experts (MoE) built for mission-critical environments, released on October 3, 2026 under an Apache 2.0 license with FP8 weights. If large models remain a data-center affair, MoE is the most concrete way to bring that quality onto hardware that fits in an office. In this article we put it on the bench: what it takes to run it locally, what it really costs, and what is still missing.

Kolibri in one table

The official numbers from the vendor site and the model card on Hugging Face:

Spec	Value
Architecture	MoE: 50 layers, 384 experts per layer + 1 shared, top-6
Total parameters	78.1B (78,103,074,560)
Active parameters per token	3.46B (3,457,573,120)
Context length	1,048,576 tokens (262,144 or less recommended)
Precision	FP8 for weights and KV cache, BF16 for embeddings and router
Specialized languages	German and English
License	Apache 2.0, open weights
Release date	October 3, 2026
Declared minimum	2x A100 80GB, 2x H100 SXM5, 1x H200, 1x B200 or 1x B300

The model was trained on 20 trillion tokens of a bilingual corpus (~62.5% English, ~23.9% German, ~13.6% code) on 768 B200 GPUs in 21 days; the knowledge cutoff is June 18, 2026 for both languages. Sources: official page and model card.

MoE: 78B of quality, 3.46B of cost

In a dense model, every token passes all parameters through the computation. In a Mixture-of-Experts, each layer holds hundreds of "experts" (smaller sub-networks) and a router activates only a few: in Kolibri, 6 out of 384 per layer, plus one always-on shared expert. The result is the memory capacity of a 78B model and the compute cost of a ~3.5B model.

Why it matters: generation cost scales with active parameters, not total ones. The total parameters must fit in VRAM (or the model does not start), but each token is computed by a small fraction of them. It is the same principle as the models you already know — Qwen3 30B-A3B or Gemma 4 26B-A4B — pushed onto a bigger architecture.

On the benchmarks published by the vendor (their own measurements: read them with that lens):

Benchmark	Kolibri	Qwen3.6 35B-A3B	Nemotron 3 Super 120B-A12B
GPQA Diamond (EN)	84.3%	83.4%	78.0%
AIME 2026 (EN)	96.0%	91.0%	90.4%
Tau3-Bench (Banking)	38.1%	10.6%	15.5%
BFCL v4	61.4%	67.2%	61.0%

Two readings: on reasoning and math it sits with models that are far larger in active parameters; on agentic tool calling (Tau3-Bench Banking) it is clearly above MoE of its class, but behind Qwen3.6 35B-A3B on BFCL. On the vendor's efficiency chart, Kolibri scores a 71% average on the German benchmarks at ~47,000 bytes/s per GPU, on the frontier of the curve.

The local test: what you need

This is where it gets interesting — and where it gets complicated.

The official weights are FP8, with a declared footprint of ~78GB of model memory: the vendor's minimum is two 80GB GPUs. The community has already produced the GGUF conversions: the Hob-forge/Kolibri-1-GGUF repository (an independent conversion, not made or endorsed by Aleph Alpha) offers quantizations from 2 to 8 bits:

Quantization	Size	Realistic hardware
Q2_K	28.6 GB	RTX 5090 32GB (~$5,220), Mac with 48GB unified
Q3_K_M	37.5 GB	2x 32GB GPUs, Mac with 64GB unified
Q4_K_M	47.5 GB	96GB GPU, Mac Studio M5 Max 128GB (~$5,099)
Q5_K_M	55.6 GB	2x 32/48GB GPUs, Mac with 96GB+ unified
Q8_0	83.1 GB	2x 80GB GPUs (nearly the original weights)

The price you pay: stock llama.cpp does not support the kolibri1 architecture yet. You apply the patch included in the GGUF repo (on llama.cpp commit 836d571) and compile, or use the prebuilt binaries the same community released. Ollama and LM Studio will not load it until they add support.

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git checkout 836d571
git am /path/to/kolibri1-llama.cpp.patch
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-cli
./build/bin/llama-server -m Kolibri-1-Q4_K_M.gguf -c 32768 --jinja --temp 1.0 --top-p 0.97 --top-k 128

When it works: open http://localhost:8080 in your browser and the chat is live, with the reasoning mode controllable per request (none, low, medium, high). When it does not: check the patch was applied on the right commit, and that VRAM covers weights + KV cache — a 32K context adds several gigabytes on top of the model weights.

The math: what it really costs

Two configurations, from the list-price dataset we use in the setup planner (October 2026):

Setup	Price (2x if dual)	36-month amortization
2x H100 80GB SXM (vendor minimum, FP8)	2 x $27,000 = $54,000	$1,500/month
Mac Studio M5 Max 128GB (Q4_K_M)	$5,099	$142/month

The electricity bill, for the data-center setup, at 10 hours a day and €0.30/kWh: two 700W H100s make 1.4 kW under load; 1.4 x 10 = 14 kWh per day, i.e. €4.20 per day, ~€1,530 per year. The Mac Studio's bill is an order of magnitude lower: the unified machine draws a fraction of the watts of two 700W cards.

Setup	36-month amortization	36-month electricity (10h/day)	36-month total
2x H100 80GB	$54,000	~$5,000	~$59,000
Mac Studio M5 Max 128GB (Q4_K_M)	$5,099	far less	~$5,500

The 10:1 ratio is in the cards, not the bill. And: on your own hardware, the AI is always on — no token fees, no usage caps, data stays home. On the cloud, cost scales with your usage: if you generate 10 hours a day, local pays back in months; if you use it one hour a day, the cloud can cost less for a whole year.

And the limits?

• Languages: German and English, full stop. The corpus is 62.5% English and 23.9% German: for work in other languages, Qwen or Gemma deliver more reliable results today. Kolibri is a serious European model, but it is not a multilingual one.
• Friction: a patch to compile, no Ollama, no LM Studio. For the "works in 10 minutes" flow, Kolibri's time has not come yet.
• VRAM: even at Q4_K_M it is 47.5GB. 16GB cards are out of the game.
• Benchmarks: all vendor numbers; independent measurements on local hardware are still scarce.
• 1M context: vendor-validated, but the recommendation is 262K tokens or less for efficiency and quality on complex tasks.

Where to start

To understand MoE before committing hardware: when a big model is not what you need. To size your RAM: KV cache: why RAM matters more than the model. And for the multilingual path: Qwen, the open model with the best multilingual support.

In summary

Kolibri is the European open-weight MoE of the autumn: 78.1B parameters with 3.46B active per token, context up to 1M (256K recommended), FP8 weights, Apache 2.0, reasoning mode and tool calling. The vendor benchmarks put it on top for reasoning and math, near the efficiency frontier; the real cost is VRAM (~78GB in FP8, 47.5GB in Q4_K_M) and the llama.cpp patch friction that Ollama and LM Studio have not landed yet. For other languages, multilingual versions are still to come: today the path is Qwen. When the kolibri1 architecture lands in llama.cpp and Ollama, Kolibri will be the first model to put through its paces on your hardware.

Article generated with AI