andysalerno/r9700-serving

The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper

A high-throughput and memory-efficient inference and serving engine for LLMs - Optimized for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60

Codeberg fork of StillDeadcode / vllm-radiance which is a custom vllm tuned for R9700 GPU's

Working on the fastest vllm path for AMD Radeon AI PRO R9700

Portable vLLM builds with AMD ROCm acceleration for Lemonade

Configuration and documentation for optimizing Ubuntu 24.04 on AMD Ryzen AI Max+ 395 with Radeon 8060S for LLM inference using llama.cpp with ROCm 7 RC and rocWMMA.