Skip to main content
← Back to news

llama.cpp b11717: Hadamard for Q1_0 and Q2_0 quantizations on Snapdragon NPU

Hardware & accelerators
Source:GitHub
Build b11717 adds Hadamard MUL_MAT and GET_ROW operations for Q1_0 and Q2_0 quantizations (flat and tiled) to the Hexagon backend (Snapdragon NPU), and reduces packet counts in the q8_1 quantizer. Qualcomm contribution: a further step toward mobile NPU Inference for the most aggressive quantizations.
Read original article

Related articles

KV cache: why your RAM needs more than just the model

What the KV cache is, the formula to calculate it with real numbers from Llama 3.1 8B and Qwen3 8B, and the three levers to manage it: context, quantization and flash attention.

Llama 3 and Llama 4: Meta's open family — which model runs on your hardware

From the 3.1 8B to the 4 Scout MoE with a 10M context: the Llama family in a table, what runs on 16, 48 and 128 GB of RAM, the Community license translated for the Italian business and the choice by budget.

What a token costs: five routes, the math, and the break-even point

Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🦙

Llama 3.1 405B

Models

Meta's largest open model: closed-frontier quality but datacenter requirements. In practice it needs high-VRAM multi-GPU or the cloud.

🦙

Llama 3.1 70B

Models

Frontier-class open model with 131K-token context. Needs 48-80GB VRAM, a Mac with unified memory, or a multi-GPU setup.

🦙

Llama 3.1 8B

Models

The de-facto local workhorse: solid quality, 131K-token context and a huge GGUF quantization ecosystem. The standard choice for chat and assistants.

🦙

Llama 3.2 1B

Models

Small, fast model with up to 131K-token context. Ideal entry point for chat and summarization on modest hardware, great on phones and mini-PCs.

Browse all resources