causalflow-ai/petit-kernel

FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes of 16-32 tokens.

ROCmFPX Family for AMD Hardware and Processors. More quants and special agent quants

Profile-guided GPU kernel optimizer for AMD/RDNA3. Auto-tunes llama.cpp MMVQ kernels per model shape. 2x decode speedup on 7900 XTX.

A high-throughput and memory-efficient inference and serving engine for LLMs - Optimized for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60

DeepGEMM: clean and efficient BLAS kernel library on GPU

An agent harness that compiles a model into one provably-correct, self-retargeting CUDA megakernel and self-tunes it past cuBLAS at batch-1 LLM decode, paper: https://arxiv.org/abs/2606.09682

RDNA-native LLM inference engine in Rust.

FlashInfer: Kernel Library for LLM Serving

The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper