sebastianmechno-sys/vllm-rocm-windows-rdna2

WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.

A high-throughput and memory-efficient inference and serving engine for LLMs - Optimized for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60

Run LLMs on AMD Vega8 / Vega10 APUs and GPUs (Ryzen 5700G / gfx90c and Vega 56/64): ROCm 7 gfx900 backport + Vulkan/RADV launchers, Docker images, benchmarks for llama.cpp and LM Studio

Fresh builds of llama.cpp with AMD ROCm™ acceleration

RDNA-native LLM inference engine in Rust.

Native Windows vLLM: 0.27.1 Latest; 0.29.0 Python 3.14/CUDA 13.2 release (CPU TorchAudio), plus Python 3.13/CUDA 13.0 prerelease. PyTorch 2.13, FlashAttention/Rust, OpenAI serving, 10 KV formats, Multi-TurboQuant, experimental CPU/NVMe KV offload. No WSL or Docker.

The fastest vLLM build for AMD R9700's. For the latest updates join the Launch80 discord server. I am on X now also https://x.com/bkuyper

Codeberg fork of StillDeadcode / vllm-radiance which is a custom vllm tuned for R9700 GPU's

This will be the official ROCmFPX stack and studio