🚀 LLM inference Engine in Swift/Metal, Load GGUF and safe tensors modes, no conversion, no cpp, pure swift
Like christopherkarani/EdgeRunner, 0 likes 0 Bookmark christopherkarani/EdgeRunner, 0 bookmarks 0 Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 276B, native Mac app, CLI, and OpenAI-compatible server.
Like NeelM0906/Mference, 0 likes 0 Bookmark NeelM0906/Mference, 0 bookmarks 0 High-performance MLX-based LLM inference engine for macOS with native Swift implementation
Like Trans-N-ai/swama, 0 likes 0 Bookmark Trans-N-ai/swama, 0 bookmarks 0 lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.
Like lithos-ai/lithos-metal, 0 likes 0 Bookmark lithos-ai/lithos-metal, 0 bookmarks 0 ⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
Like SharpAI/SwiftLM, 0 likes 0 Bookmark SharpAI/SwiftLM, 0 bookmarks 0 fastest runtime for apple silicon.
Like SRSWTI/bodega-inference-engine, 0 likes 0 Bookmark SRSWTI/bodega-inference-engine, 0 bookmarks 0 Run LLMs on Apple devices with CoreML, optimized for Apple Neural Engine + GPU
Like john-rocky/CoreML-LLM, 0 likes 0 Bookmark john-rocky/CoreML-LLM, 0 bookmarks 0 Our lightning-fast inference engine.
Like recursive-oss/rec, 0 likes 0 Bookmark recursive-oss/rec, 0 bookmarks 0 Fastest LLM inference runtime for Apple Silicon
Like basecompute/baseRT, 0 likes 0 Bookmark basecompute/baseRT, 0 bookmarks 0