Tools, frameworks, and models for local AI
3 resources found for "Kernel"
27B reasoning model with ternary weights at 1.72 bits per weight, derived from Qwen3.8-27B with hybrid attention (~75% linear). In GGUF PTQ1_0 it takes 5.95 GB, retains 98.2% of the FP16 intelligence and reaches ~47 tok/s on an Apple M5 Max laptop, with a 262K-token context. Apache 2.0 license. Requires Prism ML's llama.cpp fork (ternary kernels are not in stock llama.cpp).
NVIDIA's Rust CLI for AI-assisted Linux kernel development, focused on local AI: patch validation, builds, boot and tests, designed for backporting patches to older kernels. It supports local back-ends (Claude, OpenCode, Codex) or OpenAI-compatible servers. Apache 2.0, led by Andrea Righi.
Open-source inference engine (Rust, YC S25) that compiles and tunes its kernels directly on the target hardware: Apple Silicon, NVIDIA, AMD or CPU. On open models it claims up to 2× the speed of llama.cpp. Also aimed at serving agents.
