Tools, frameworks, and models for local AI
50 resources found for "Local"
AMD: RDNA-architecture Radeon GPUs are the second consumer platform for local AI. Since ROCm 7, consumer cards (RX 7900 XTX, RX 9070 XT) are officially supported on Linux with no workarounds, and the Strix Halo APU brings up to 128GB of unified memory in a laptop. The GPUs and APUs in the catalog are the child resources.
La Radeon con la maggiore capacità consumer: 24GB GDDR6 a 960 GB/s (bus 384-bit). Per l'IA locale la capacità batte la novità: mentre le schede da 16GB non caricano i modelli 27B, qui Qwen3.8-27B e i 32B in Q4 ci stanno con contesto ampio. 960 GB/s è il 96% della RTX 4090. ROCm ufficiale (gfx1100), 355W.
Apple Inc. (Cupertino, USA): Apple Silicon chips and unified-memory Macs are a reference for local AI on consumer hardware. Apple products in the catalog are the child resources.
Il chip Apple Silicon di punta per l'IA locale: 18 core CPU e 40 core GPU con memoria unificata fino a 128GB a 614 GB/s. Il Neural Accelerator su ogni core GPU raddoppia la velocità dei modelli MoE in decode rispetto alla generazione M4: ~88 tok/s su gpt-oss-120b, il più veloce della classe.
Bespoke Labs' 9B decision model: it runs entirely locally via Ollama 0.35 and answers typed questions (true/false, numeric scale, multiple choice) through the new systemone API. Built for fast classification in automated pipelines, no cloud.
NVIDIA's Rust CLI for AI-assisted Linux kernel development, focused on local AI: patch validation, builds, boot and tests, designed for backporting patches to older kernels. It supports local back-ends (Claude, OpenCode, Codex) or OpenAI-compatible servers. Apache 2.0, led by Andrea Righi.
DeepSeek's MoE model with 552B total parameters and 8-16B active per token. 1M-token context, MIT license: free commercial use. Built for online serving, too large for local use on a single PC.
La taglia 12B della famiglia Gemma 4 di Google: 11,95B dense, contesto 256K e multimodalità integrata senza encoder separati (testo+immagini+audio). Licenza Apache 2.0. In Q4_K_M occupa circa 7,4GB: il punto di equilibrio tra qualità e requisiti hardware per l'uso locale su 8-16GB.
Compact model from Google's Gemma 4 family: 8B total parameters, of which 4.5B effective thanks to per-layer embeddings (PLE). 131K-token context, 140+ languages, text+image and audio input. The entry size of the Gemma 4 family for local use on 8-16GB.
Hugging Face's Python library for running the pipelines of generative models: image, video, and audio. The standard way to run models like Qwen-Image locally, with CUDA support and CPU offload for lower-memory GPUs.
The cloud platforms for models: alternatives when local hardware isn't enough. Replicate and Runpod in the catalog are the child resources.
Intel: Arc (Xe) GPUs are the third consumer platform for local AI, with 12-16GB of VRAM at entry prices. The maintained inference path is Vulkan (native Ollama, llama.cpp): the IPEX-LLM project was archived in January 2026. The GPUs in the catalog are the child resources.
La prima scheda Intel da prendere sul serio per l'IA locale: Xe2 Battlemage, 12GB GDDR6 a 456 GB/s, 190W. 12GB a prezzo d'ingresso: i modelli 8-14B in Q4 stanno interamente in VRAM. Il software è il punto debole: IPEX-LLM è stato archiviato a gennaio 2026, il percorso mantenuto oggi è Vulkan (Ollama nativo, llama.cpp) con ~40-42 tok/s su 8B Q4.
Aleph Alpha's MoE: 78.1B parameters (3.46B active per token), Apache 2.0, FP8 weights, context up to 1M tokens, reasoning mode and tool calling. Built for mission-critical workloads on local hardware.
The de-facto local workhorse: solid quality, 131K-token context and a huge GGUF quantization ecosystem. The standard choice for chat and assistants.
C++ implementation of LLMs for efficient inference on consumer CPUs and GPUs.
Open-source tool that estimates local model inference speed across 250+ hardware configurations (GPU, Mac, CPU): measured estimates from community benchmarks or derived from similar setups. Use it to choose hardware before buying.
Terminal-free desktop app for downloading and running GGUF models: GUI, built-in model search and a local server. The simplest way to get started.
An online guide matching open-weights models to real hardware: for every GPU, Mac and mini-PC it shows what runs, how much memory it needs and at how many tokens per second, with guides on quantization and memory. The right tool for the own-hardware path.
Il desktop compatto di Apple con il chip M6, primo processore Apple in 2nm: 12 core CPU, 12 core GPU con Neural Accelerator in ogni core e Neural Engine doppio da 16 core. 16GB di memoria unificata di serie (fino a 32GB, 170 GB/s), Wi-Fi 7, Thunderbolt 5. La piattaforma d'ingresso per IA locale: modelli fino a ~20B quantizzati a 4 bit girano interamente sul dispositivo, e la versione M5 Pro arriva a 64GB e 307 GB/s.
Il Mac più potente di Apple: M5 Ultra, primo chip a quattro die (due M5 Max uniti da UltraFusion), fino a 36 core CPU, 80 core GPU e 512GB di memoria unificata a 1,2 TB/s. Pensato per modelli frontier-class in locale: MoE da centinaia di miliardi di parametri e RAG su corpora enormi, senza cloud. Il clustering RDMA di più unità porta fino a 3× l'inferenza distribuita.
Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.
Mistral AI (Paris, France): the most efficient European open-weight LLMs, with permissive licenses and a focus on privacy and localization. The Mistral models in the catalog are the child resources.
Open-source proxy that intercepts and displays the contents of HTTPS requests in plaintext: the tool to verify what actually goes over the network from your local AI setup.
GTK4/libadwaita app for GNOME to chat with local models via Ollama, in Rust under GPLv3: it handles Ollama setup, models, and loading local documents and images.
The runtimes that run local models on your hardware: Ollama for simplicity, llama.cpp for portability, vLLM for production throughput. The engines in the catalog are the child resources.
Self-hosted workflow automation platform: a visual node-based canvas to connect apps, data and AI models (local or API), with AI Agent nodes and over 400 integrations.
NVIDIA: CUDA-architecture GPUs are the de facto standard for local AI inference. VRAM determines which models run locally: 24GB (RTX 4090) covers ~30B quantized, 80GB (A100/H100) covers 70B+ models at full precision. The GPUs in the catalog are the child resources.
La workstation desktop NVIDIA per l'IA locale: superchip Grace Blackwell GB10 con 128GB di memoria unificata a 273 GB/s e circa 1 petaFLOP in FP4. CUDA completo, ~12 tok/s su modelli 27B a 4 bit, prefill veloce su contesti lunghi. Prezzo al lancio 3.999 USD, poi 4.699 USD per la carenza di memoria.
A 500-million-parameter foundation model from NVIDIA for tabular data: prediction, imputation and generation on structured datasets, with OpenMDW 1.1 support and ready-to-use checkpoints. An example of local AI on business data, not text generation.
Modello di sintesi vocale multilingue di NVIDIA: 357M parametri, 12 lingue (italiano e francese inclusi), architettura encoder-decoder transformer. Compatto e pensato per esecuzione in locale a bassa latenza; pesi disponibili su Hugging Face.
NVIDIA's vision encoder-decoder family for local document understanding: markdown, LaTeX formulas, bounding boxes and a semantic class per element, in reading order. ViT-H (C-RADIO) + 10-block mBART decoder, all under 1 billion parameters. Locally via vLLM (v0.20–v0.26 for 2.0) or Transformers, on NVIDIA GPUs (Turing through Blackwell).
The most accessible consumer GPU with 12GB VRAM: the entry point for local inference. Comfortably runs 7-8B quantized models.
The world's most powerful consumer GPU for local training and high-speed inference.
The fastest consumer GPU on the planet: 32GB GDDR7 and 1,792 GB/s bandwidth. The top pick for local inference without going to the datacenter.
Full ChatGPT-style web interface for Ollama and llama.cpp: multi-user accounts, organized conversations, document management for RAG, per-model system prompts, tools and plugins. Version 0.11.4 (September 2026) makes file metadata travel with the retrieved chunks and runs hybrid search natively in the database: large knowledge bases are much faster. The simplest way to turn your local AI into an everyday product.
Sintesi vocale in locale a bassa latenza basata su ONNX: genera parlato naturale da testo in tempo quasi reale, anche su CPU. Multilingue (italiano incluso) e facile da integrare nei pipeline: download del modello, una riga di comando. MIT.
7B open-weight model (32-layer DiT) that unifies text-to-image generation up to 2048x2048, editing with up to 10 reference images, and native RGBA transparency. Designed to run locally on 16GB+ VRAM; supported by diffusers and ComfyUI.
Ultra-compact 0.5B-parameter model. Runs on any hardware, even embedded: ideal for lightweight summarization and low-power local devices.
Il modello dense 8B della famiglia Qwen3: 8,19B di parametri, contesto nativo di 40K token (espandibile con YaRN) e ragionamento in modalità think/non-think attivabile per prompt. Licenza Apache 2.0. È il punto di equilibrio per l'uso locale su GPU da 8-12GB: in Q4_K_M occupa circa 5GB, lasciando spazio per contesto lungo.
0.6B safety-moderation model with a token-level classifier for real-time monitoring during incremental generation. Three severity tiers (safe, controversial, unsafe) and 119 languages: small and local, ideal for filtering outputs without the cloud.
Single-board computer with an 8-core ARM Cortex-A76 CPU and 2–16 GB of RAM: the reference for learning and running small local AI models in CPU inference.
A zero-dependency Python tool that recognizes and summarizes every file with a local LLM (Ollama, LM Studio or MLX-LM), fully offline. Detects type by content, extracts structure (CSV, email, scripts, logs) and reports files carrying instructions to the assistant. Works without a model (--no-llm). MIT license.
Up to 47 seconds of text-to-audio: high-quality music and sound effects. Excellent for local audio production and sound design.
Turns an image into a short ~2s video clip. The entry point for local video generation; 16GB VRAM recommended.
The graphical frontends for using local models without accounts or cloud uploads: LM Studio, ComfyUI, Fooocus and Open WebUI. The studio in the catalog are the child resources.
100% local macOS dictation app: on-device transcription with Whisper and a rewrite pass with a local LLM (Qwen) on the M-series GPU. Text lands in the target app; no data touches the cloud.
Il modello di trascrizione di riferimento per l'IA locale: 809M di parametri, 99 lingue, licenza MIT. Versione distillata di Large-v3 con decoder ridotto da 32 a 4 layer: trascrive circa 8 volte più veloce con perdita di accuratezza trascurabile. Gira in tempo reale su una GPU consumer e anche solo su CPU.
Il port C/C++ ad alte prestazioni di Whisper: trascrizione vocale in locale ottimizzata per CPU e Apple Silicon, senza dipendenze da Python. Quantizzazione GGML (Q4_0–Q8_0) con riduzione fino al 65% a disco, streaming via whisper-stream, accelerazione Metal/Vulkan. MIT.
