ROCm/ATOM

Pure Rust Inference Engine

RDNA-native LLM inference engine in Rust.

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

LLM inference in C/C++

A highly optimized LLM inference acceleration engine for Llama and its variants.

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper

The inference engine the open-source world built for itself.