christopherkarani/EdgeRunner

High-performance MLX-based LLM inference engine for macOS with native Swift implementation

Run LLMs on Apple devices with CoreML, optimized for Apple Neural Engine + GPU

LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.

Swift package to run local LLMs on iOS, macOS, Linux

Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage, enabling 35B and 80B models to run with low RAM, including on iPhone.

Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 276B, native Mac app, CLI, and OpenAI-compatible server.

Large Language Model (LLM) module for the Spezi Ecosystem

lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.