videlalvaro/ane-book

Artificial Neural Engine Machine Learning Library

Run LLMs on Apple devices with CoreML, optimized for Apple Neural Engine + GPU

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

Everything we actually know about the Apple Neural Engine (ANE)

Reference implementation of the Transformer architecture optimized for Apple Neural Engine (ANE)

Fastest LLM inference runtime for Apple Silicon

🚀 LLM inference Engine in Swift/Metal, Load GGUF and safe tensors modes, no conversion, no cpp, pure swift

Swift + Metal MoE inference for Apple Silicon: Qwen 3.6 35B at 23.5–29.3 tok/s decode with 2.20× faster long-prompt prefill on a 24 GB M5; Gemma 4 26B in ~2 GB, DeepSeek-V4-Flash 284B, Inkling-Small 276B, native Mac app, CLI, and OpenAI-compatible server.

Run transformers directly on Apple's Neural Engine in Swift bypass coreml entirely