AdamBien/lightmetal

lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.

A rust mlx server, ui, and model router for fast, dependency free inference on apple architecture

Our lightning-fast inference engine.

Pure C++23 LLM inference for Apple Silicon chips

Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 276B, native Mac app, CLI, and OpenAI-compatible server.

Fastest LLM inference runtime for Apple Silicon

fastest runtime for apple silicon.

macMLX brings local LLM inference to Apple Silicon with a first-class native macOS experience. No cloud, no telemetry, no Electron — just your Mac running models at full speed.

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon