High-speed Large Language Model Serving for Local Deployment
Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory
A fast inference library for running LLMs locally on modern consumer-class GPUs
Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon
Python package wrapping llama.cpp for on-device LLM inference
A high-performance, GGUF-native Rust & CUDA inference engine optimized for cold-start latency and real-time 'System 1' agent decision loops.
Turn any Mac or GPU into an OpenAI-compatible inference node. One-command setup, automatic HTTPS, model management, and distributed request routing.
An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs
Run local AI models privately with GPU acceleration. Modern UI, HuggingFace integration. Built with Tauri + React + Rust.
