carloslfu/slotstream

High-performance MLX-based LLM inference engine for macOS with native Swift implementation

Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 276B, native Mac app, CLI, and OpenAI-compatible server.

Running a big model on a small laptop

Run GPT-OSS-120B on a 16 GB Mac. Streams mixture-of-experts weights from SSD so only the 2 GiB trunk stays resident. Swift + Metal, no dependencies.

Fork of antirez/ds4 for Apple Silicon, tuned mostly for M1/M2: Qwen3.8-Flash-Next with 262K context on 64 GB and up, 35 tok/s on an M1 Max, MTP and vision, one-line install.

DeepSeek-V4-Flash-0731 284B inference in ~25 GB of RAM / Qwen3.8-Next-Flash inference in ~15 GB of RAM on any M-series MacBook

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

macMLX brings local LLM inference to Apple Silicon with a first-class native macOS experience. No cloud, no telemetry, no Electron — just your Mac running models at full speed.