Inference⚡
llama.cpp
Released Oct 15, 2022
C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.
LLMC++LocalEfficient
Back to resources
Inference⚡
faster-whisper
Released on Feb 11, 2023
Reimplementation of Whisper based on CTranslate2: up to 4 times faster than openai/whisper with the same accuracy and less memory. Supports 8-bit quantization on both CPU and GPU.
Features
- CTranslate2, up to 4x faster
- 8-bit quantization (CPU and GPU)
- Python, MIT
- 99 Whisper languages
Version history
- v1.2.1Latest
Ultima release verificata: mantenute le ottimizzazioni CTranslate2 e la quantizzazione 8-bit su CPU e GPU.
- v1.1.1
Serie 1.x: API stabile per il modello Whisper, supporto ai modelli large-v3 e tooling per la trascrizione in streaming.
- v0.2.0
Prima release taggata su PyPI: reimplementation di Whisper su CTranslate2 con benchmark pubblici rispetto a openai/whisper, whisper.cpp e transformers.
Tags
Useful links
Discover faster-whisper
Visit the official site to download, learn more and start using faster-whisper.
Related resources — Inference
Inference🦙
Ollama
Released Jul 27, 2023v0.35.0
Open source tool to run LLMs locally with simplicity.
LLMLocalOpen SourceCLI
Inference🗣️
Piper TTS
Released Apr 1, 2022Piper
Sintesi vocale in locale a bassa latenza basata su ONNX: genera parlato naturale da testo in tempo quasi reale, anche su CPU. Multilingue (italiano incluso) e facile da integrare nei pipeline: download del modello, una riga di comando. MIT.
AudioTTSOpen SourceLocale