zolotukhin/zinc

LLM Inference Engine for Metal, CUDA and Vulkan.

Our lightning-fast inference engine.

High-speed Large Language Model Serving for Local Deployment

Fastest LLM inference runtime for Apple Silicon

Big models. Small Macs. Zero excuses.

Atlas Inference is a dependency free lightweight inference engine meant to run LLMs on YOUR compute seamlessly. Supports GB10 and Strix Halo at the moment, hoping to expand to more!

Open source inference engine that tunes its kernels on your hardware to find the fastest way to run each model. 2x faster than Ollama. Works on Apple Silicon, NVIDIA, AMD, Intel, or just a CPU.

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

lithos-metal: high-performance LLM inference kernels and serving for Apple silicon.