Skip to main content
← Back to news

LLMspeed: test how fast your local model will be before you buy it

Tools & releases
Source:Hacker News
LLMspeed is an open-source tool (Hacker News showcase) that estimates Inference speed for 250+ model/hardware combinations: RTX 3060 to H100, Mac Studio M5 Ultra, CPU-only. Measured estimates come from community benchmarks (r/LocalLLaMA, localscore.ai); the rest are derived from similar setups or projected from specs. Useful for choosing hardware before spending.
Read original article

Related articles

GPU or CPU: when the CPU is enough

Do you really need a GPU for local AI? For many use cases, no: how prefill and decode work, what matters (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure on your own hardware.

Mac Studio M5 Ultra: 512GB of unified memory on the desktop

Four dies, 512GB of unified memory at 1.2 TB/s: the Mac Studio M5 Ultra runs 100-560B MoE models locally. The speed math, the cost comparison with the DGX Station, and when the Mac is the right choice.

Unified memory: from Strix Halo to Mac Studio M5 Ultra — what fits and at what speed

128GB or 512GB of memory shared between CPU and GPU: three real platforms, the bandwidth that determines speed, and the math on what fits and for whom it makes sense.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🖥️

NVIDIA RTX 3060 12GB

Hardware

The most accessible consumer GPU with 12GB VRAM: the entry point for local inference. Comfortably runs 7-8B quantized models.

🍓

Raspberry Pi 5

Hardware

Single-board computer with an 8-core ARM Cortex-A76 CPU and 2–16 GB of RAM: the reference for learning and running small local AI models in CPU inference.

⚡

LLMspeed

Software

Open-source tool that estimates local model inference speed across 250+ hardware configurations (GPU, Mac, CPU): measured estimates from community benchmarks or derived from similar setups. Use it to choose hardware before buying.

🖥️

Mac Studio (M5 Ultra)

Hardware

Il Mac più potente di Apple: M5 Ultra, primo chip a quattro die (due M5 Max uniti da UltraFusion), fino a 36 core CPU, 80 core GPU e 512GB di memoria unificata a 1,2 TB/s. Pensato per modelli frontier-class in locale: MoE da centinaia di miliardi di parametri e RAG su corpora enormi, senza cloud. Il clustering RDMA di più unità porta fino a 3× l'inferenza distribuita.

Browse all resources