jrz97619761/test-model-thing

Exact speculative decoding on Apple Silicon, powered by MLX.

Run LLMs with MLX

A native macOS lab for teaching tiny language models to think — build the architecture, train the weights, and watch a small LLM emerge from scratch, locally on Apple Silicon with custom data, tokenizers, checkpoints, and MLX acceleration.

Our lightning-fast inference engine.

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

Run large Mixture-of-Experts language models well on the memory you have.

Lossless DFlash speculative decoding for MLX on Apple Silicon

Low-level Metal inference engine for Apple M4: universal quant formats at MLX parity, 1M-token out-of-core contexts, and speculative decode off flash storage.

works well with rocks; native Apple Silicon inference engine for capable and tiny models (ternary, Qwen) via MLX Swift. Hadamard-rotated KV cache, vision, tool calling, OpenAI + Anthropic APIs