Google releases ML Drift, an open-source on-device GPU inference engine
On-device & edgeRelated articles
What a token costs: five routes, the math, and the break-even point
Frontier API, subscription, inference providers, rented GPU or your own hardware: five routes to tokens, with real monthly costs, the electricity bill, and the calculated break-even point.
GPU or CPU: when the CPU is enough
Do you really need a GPU for local AI? For many use cases, no: how prefill and decode work, what matters (memory bandwidth), real numbers on a mainstream laptop, the cases where the CPU is enough and where VRAM is needed — with the commands to measure on your own hardware.
vLLM: serving models fast
vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the concepts (throughput, quantization, KV cache) for choosing when to use it instead of Ollama.
Get the next local-AI briefing
Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.
After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.