RWKV/rwkv.cpp

Inference RWKV with multiple supported backends.

RWKV-7 7.2B fp16 15000+ tps decoding @ single 5090

Inference RWKV v5, v6 and v7 with Qualcomm AI Engine Direct SDK

A minimal MiniCPM5-1B model inference engine built purely in Rust

llama.cpp fork with additional SOTA quants and improved performance

Experimental llama.cpp fork for inference research and development

NInfer port for Qwen 3.8 27B on RTX 4090

Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.