Skip to main content
← Back to news

Sriti Core: a local-first LLM router that cascades from Ollama to the cloud

Tools & releases
Source:Hacker News
An open-source routing proxy (Apache 2.0) that cascades LLM requests: it starts with the local model on Ollama and climbs to the cloud only when needed, with semantic caching and per-hop reliability tracking. An interesting pattern for those who want local by default.
Read original article

Related articles

Running LLM Models Locally: From First Install to First Working Model

From zero to a first answering model: what you actually need, Ollama step by step, how to pick a model for your memory, expected speeds per hardware, and the typical problems fixed.

Qwen: the open model with the best multilingual support

Qwen is the open family with the best support for Italian and French: generations, sizes, MoE, when to use it, and a quick comparison with Llama.

vLLM: serving models fast

vLLM is the open-source engine that turns a local model into a concurrent server: PagedAttention, continuous batching, installation and the concepts (throughput, quantization, KV cache) for choosing when to use it instead of Ollama.

Get the next local-AI briefing

Once a week, the top selection from iaLocale: articles, news and resources on local AI, and more.

After signing up you receive a confirmation email (double opt-in). Without confirmation we send nothing. No spam, we promise.

Related resources

Tools and resources from the iaLocale archive relevant to this content.

🦙

Ollama

Inference

Open source tool to run LLMs locally with simplicity.

Bespoke Nimble 9B

Models

Bespoke Labs' 9B decision model: it runs entirely locally via Ollama 0.35 and answers typed questions (true/false, numeric scale, multiple choice) through the new systemone API. Built for fast classification in automated pipelines, no cloud.

⚡

llama.cpp

Inference

C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.

⚡

Magnitude

Inference

Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.

Browse all resources