Inference⚡
llama.cpp
Released Oct 15, 2022
C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.
LLMC++LocalEfficient
Resources
Tools, frameworks, and models for local AI
3 resources found for "Llama"
Tools, frameworks, and models for local AI
3 resources found for "Llama"
C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.
Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.
Open source tool to run LLMs locally with simplicity.