Tools, frameworks, and models for local AI
17 resources found for "Hardware"
Apple Inc. (Cupertino, USA): Apple Silicon chips and unified-memory Macs are a reference for local AI on consumer hardware. Apple products in the catalog are the child resources.
Google's open model family in 1B, 4B, 12B and 27B sizes. 128K-token context, 140+ supported languages and multimedia input (text and images). An excellent choice for RAG and long documents on consumer hardware.
La taglia 12B della famiglia Gemma 4 di Google: 11,95B dense, contesto 256K e multimodalità integrata senza encoder separati (testo+immagini+audio). Licenza Apache 2.0. In Q4_K_M occupa circa 7,4GB: il punto di equilibrio tra qualità e requisiti hardware per l'uso locale su 8-16GB.
The cloud platforms for models: alternatives when local hardware isn't enough. Replicate and Runpod in the catalog are the child resources.
Aleph Alpha's MoE: 78.1B parameters (3.46B active per token), Apache 2.0, FP8 weights, context up to 1M tokens, reasoning mode and tool calling. Built for mission-critical workloads on local hardware.
Small, fast model with up to 131K-token context. Ideal entry point for chat and summarization on modest hardware, great on phones and mini-PCs.
Open-source tool that estimates local model inference speed across 250+ hardware configurations (GPU, Mac, CPU): measured estimates from community benchmarks or derived from similar setups. Use it to choose hardware before buying.
An online guide matching open-weights models to real hardware: for every GPU, Mac and mini-PC it shows what runs, how much memory it needs and at how many tokens per second, with guides on quantization and memory. The right tool for the own-hardware path.
Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.
Serverless platform for running code and AI models on GPUs: you pay for real seconds of use, it scales to zero when idle, and you can run any open-weights model. One of the routes to inference on rented hardware without managing servers.
The runtimes that run local models on your hardware: Ollama for simplicity, llama.cpp for portability, vLLM for production throughput. The engines in the catalog are the child resources.
Ultra-compact 0.5B-parameter model. Runs on any hardware, even embedded: ideal for lightweight summarization and low-power local devices.
3B-parameter model with excellent multilingual support, including Italian. Good balance of quality and hardware requirements for summarization and drafting.
A 35B-parameter MoE model with ~3B active per token from the Qwen family, Apache-2.0. It is the fine-tune base of JEV-35B. Suited to 40GB+ GPUs in BF16 or to 4/8-bit quantizations for smaller hardware.
Single-board computer with an 8-core ARM Cortex-A76 CPU and 2–16 GB of RAM: the reference for learning and running small local AI models in CPU inference.
A marketplace for renting GPUs: GPU owners put their capacity up for sale and rent it out by the hour or by period, at prices that follow demand. A middle path between frontier cloud and your own hardware.
High-throughput LLM serving with PagedAttention and hardware optimizations.
