Inference⚡
llama.cpp
Released Oct 15, 2022
C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.
LLMC++LocalEfficient
Back to resources
Inference🧭
OpenRouter
Released on Mar 8, 2023
Cloud gateway for LLM inference: a single API endpoint for hundreds of models (including open-weight MoE), with per-token pricing and automatic routing across providers. For Ling 3.1 Flash: $0.075/M input, $0.225/M output.
Features
- Single API endpoint
- Multi-provider routing
- Per-token pricing
- Open-weight models
Tags
Useful links
Discover OpenRouter
Visit the official site to download, learn more and start using OpenRouter.
Related resources — Inference
Inference🦙
Ollama
Released Jul 27, 2023v0.35.0
Open source tool to run LLMs locally with simplicity.
LLMLocalOpen SourceCLI
Inference🗣️
Piper TTS
Released Apr 1, 2022Piper
Sintesi vocale in locale a bassa latenza basata su ONNX: genera parlato naturale da testo in tempo quasi reale, anche su CPU. Multilingue (italiano incluso) e facile da integrare nei pipeline: download del modello, una riga di comando. MIT.
AudioTTSOpen SourceLocale