huggingface/grout

Qwen3-0.6B megakernel: 527 tok/s decode on RTX 3090 (3.8x faster than PyTorch)

Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux builds.

NInfer port for Qwen 3.8 27B on RTX 4090

From-scratch C++/CUDA inference engine for Qwen3-8B

High-performance single-GPU inference for selected model checkpoints and GPUs.

One line of NInfer for the RTX 3090, RTX 4090 and RTX 5090, consolidated from the forks that carry it: GGUF block formats, ternary Bonsai 2, device route profiles, MTP and DFlash2.

Perplexity open source garden for inference technology

High-performance single-GPU inference for selected model checkpoints and GPUs.

Experimental llama.cpp fork for inference research and development