quantumnic/ssd-llm

Metal-first MoE inference for Apple Silicon with bounded SSD expert streaming and local OpenAI Chat, Responses, and Anthropic Messages endpoints.

Run GreenBitAI's Quantized LLMs on Apple Devices with MLX

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

Run models too big for your Mac's memory

Run LLMs with MLX

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

Fastest LLM inference runtime for Apple Silicon

Blazing fast Apple Silicon inference engine optimized for Qwen & Nemotron. OpenAI + Anthropic API compatible. Built by folks from nace.ai

works well with rocks; native Apple Silicon inference engine for capable and tiny models (ternary, Qwen) via MLX Swift. Hadamard-rotated KV cache, vision, tool calling, OpenAI + Anthropic APIs