steadfastgaze/MoEspresso

Run LLMs with MLX

Metal-first MoE inference for Apple Silicon with bounded SSD expert streaming and local OpenAI Chat, Responses, and Anthropic Messages endpoints.

Running a big model on a small laptop

Fork of antirez/ds4 for Apple Silicon, tuned mostly for M1/M2: Qwen3.8-Flash-Next with 262K context on 64 GB and up, 35 tok/s on an M1 Max, MTP and vision, one-line install.

Run GreenBitAI's Quantized LLMs on Apple Devices with MLX

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

works well with rocks; native Apple Silicon inference engine for capable and tiny models (ternary, Qwen) via MLX Swift. Hadamard-rotated KV cache, vision, tool calling, OpenAI + Anthropic APIs

Tiny llms optimised for edge deployment

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.