MiniCPM-V 4.7: the 35B MoE that runs on a smartphone with 3 GB of RAM

35 billion parameters answering in real time on a smartphone, using 3 GB of RAM. It looks like a typo, but it's the spec sheet of MiniCPM-V 4.7, the new open model from OpenBMB and ModelBest, released on October 12, 2026. The mechanism is the one you've already met with Qwen3 30B-A3B or Gemma 4 26B-A4B — Mixture-of-Experts, MoE — brought to the phone: for every token, about 3B of the 35B total do the work. Here's what that means in practice, which numbers hold the promise, and what you need to try it.

MiniCPM-V 4.7, in one table

MiniCPM-V 4.7 is a vision-language model — multiple images and video, not just text — available in two variants: 35B-A3B, the first MoE in the series, and 1B, the compact version for even more modest hardware. The numbers from the official model card:

Spec	Value
Architecture	MoE: 35B total, ~3B active per token
Variants	35B-A3B (MoE) and 1B (dense)
Modalities	text, multiple images, video, OCR
Claimed speed	13+ tok/s on iPhone 17 Pro Max with 3 GB of RAM
Release date	October 12, 2026
Official quantizations	BF16, FP8, INT4, AWQ, GPTQ, GGUF
Engines	Ollama, llama.cpp, vLLM, SGLang
Platforms	iOS, Android, HarmonyOS; Rockchip, Intel, AMD, Arm chips day-0

The language backbone derives from Qwen 3.5/3.6 and the vision block from SigLIP2. Sources: model card and GitHub page.

MoE: 35B of capacity, 3B of work

In a dense model, every token runs all the parameters: a 35B dense costs like 35B at every step. In a Mixture-of-Experts, each layer holds many sub-models — the "experts" — and a router activates only some of them per token. For the 35B-A3B that works out to roughly 3B of compute per token: about 12 times less than a dense model of the same size (35/3 ≈ 11.7).

The problem isn't compute, it's memory: all the weights have to stay reachable, and on a phone, 35B total don't fit in RAM whole. The two classic routes have a price:

• Loading experts on the fly from storage: works, but every load makes the token wait, and the model gets slow and stuttery.
• Pinning a subset of experts for the whole prompt: faster, but you exclude the experts the model would have picked, and you lose capacity.

MiniCPM-V 4.7 goes for the third way: predict which experts will be needed and load them before they are, without touching the actual routing. That's what the three techniques below do, and the claimed result is 13+ tokens per second with 3 GB of RAM on an iPhone 17 Pro Max — the first open-source MoE to run fluidly on-device without quality compromises.

The numbers: where it fits and how fast it goes

Scenario	Format	Speed	Memory
iPhone 17 Pro Max	on-device quant (vendor)	13+ tok/s	~3 GB
Flagship Android	on-device quant (vendor)	17+ tok/s	not stated
Desktop, 24 GB GPU	GGUF Q4_K_M	to verify	21.8 GB
Desktop, 16 GB GPU	GGUF Q4_K_S	to verify	18.7 GB

Three readings. First: the 3 GB are no miracle, they're the weight actually loaded into RAM with fine per-level quantizations — the model is still 35B, the RAM holds the hot part. Second: a 21.8 GB Q4_K_M fits on a 24 GB 3090 or 4090 with room for the KV cache; on a 16 GB card you move to the 18.7 GB Q4_K_S, which is tight. Third: on the model card, the vendor shows a higher overall score than Qwen3.6 35B-A3B in less inference time — a benchmark they ran, read it with that in mind. One unofficial reference point: a user reports 37.5 t/s on an RTX 2060 Super 8 GB with llama.cpp and CPU-MoE (HF discussion) — indicative, not a vendor number.

How it gets there: three techniques, one of them vision

1. Lookahead Expert Scheduling (LES). A predictor estimates the next layer's routing and pre-loads the likely experts while the GPU or NPU computes the current layer. The prediction only guides the prefetch: the compute always follows the model's original routing, and a missed expert still arrives, loaded on the spot.

2. Fine per-level quantization. Each level of the expert weights gets its own quantization resolution, based on how much it counts. The result is less memory and less bandwidth traffic — which on a phone is often the bottleneck, more than the FLOPs.

3. DSpark speculative sampling. A small draft model (released in BF16 and GGUF) proposes several tokens at once and the big model verifies them in a single step: the same idea as speculative decoding, which you've already seen for desktop models.

And the vision part: mixed 4x/16x compression of the visual tokens inside the encoder, before they enter the language model. With 16x the model is faster — good for video and long documents; with 4x it keeps four times more tokens for fine detail — OCR, small objects. You pick at runtime, depending on the task.

And the limits?

• The 3 GB are on an iPhone 17 Pro Max, a recent flagship with plenty of RAM. On older phones or with less memory, the model slows down or doesn't fit: the vendor states no minimums for the rest of the range.
• It's a vision-language model, not fully omnimodal: images, video and OCR yes, voice no — that's the MiniCPM-o line.
• The efficiency benchmark is the vendor's, published on the model card: it compares models from other families under their own test conditions. Take it as an orientation, not a neutral measure.
• The 18.7 GB Q4_K_S on a 16 GB GPU is tight: weights, KV cache and context, the margin is thin.

Try it

1. With Ollama, the fastest route. It's in the official library:

ollama run openbmb/minicpm-v4.7

At this point you chat from the terminal. For images, hook it up to Open WebUI and attach a photo to the prompt: the model sees it. If it doesn't start, update Ollama to the latest version — multimodal models require mmproj support.

2. With Transformers, more control. On a machine with a GPU:

pip install "transformers[torch]>=5.18.0" torchvision torchcodec

Then load openbmb/MiniCPM-V-4.7-35B-A3B with torch_dtype="auto", pass downsample_mode="16x" to both apply_chat_template and generate, and use the vendor's recommended sampling parameters (temperature 0.7, top_p 0.95, top_k 100). If torchcodec won't load with your CUDA version, the documented workaround is to install av instead, or pin CUDA 12.8 in the torch install.

3. In local production. vLLM and SGLang are supported if you need a server with an OpenAI-compatible API; for consumer use, the GGUF route stays the most convenient on common hardware.

If you want the full picture of what changes with MoEs on desktop hardware, the Kolibri comparison and the JEV-35B decision-models analysis start from the same concepts at larger sizes. And to understand what happens to the weights when you compress them, the quantization guide remains the reference.

Sources

• MiniCPM-V 4.7 35B-A3B model card — Hugging Face
• Official GGUF weights
• ModelBest and OpenBMB announcement — PR Newswire, October 15, 2026
• GitHub OpenBMB/MiniCPM-V — release October 12, 2026

In summary

MiniCPM-V 4.7 35B-A3B is the first open-source MoE that runs fluidly on a smartphone without compromises: 13+ tokens per second with 3 GB of RAM on an iPhone 17 Pro Max, with roughly 3B of active compute per token. The secret is predicting which experts will be needed (LES), quantizing fine per level, and adding DSpark speculative sampling. On desktop, the 21.8 GB GGUF Q4_K_M fits on a 24 GB GPU. The on-device numbers are the vendor's, read them as such: the base is solid, but the minimums for the rest of the range need verifying on your hardware.

Article generated with AI