aivrar/vllm-windows-build

Native vLLM and ROCm 7.x runtime acceleration for AMD Radeon RX 6000 Series (RDNA2) on Windows 11.

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.

Delta Inference Engine: DeepSeek-V4.1-Flash on one NVIDIA GPU, natively on Windows (OpenAI-compatible server)

Windows port of GreenBoost - GPU VRAM extension with system RAM & NVMe for larger LLMs. Original by Ferran Duarri (GPL v2).

WHIRL (Windows HIP Inference for RDNA LLMs): native Windows C++/HIP LLM inference engine for the AMD Ryzen AI Max+ 395 (Radeon 8060S) and Radeon AI PRO R9700. Unzip and run, only the AMD driver.

High-performance single-GPU inference for selected model checkpoints and GPUs.

Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows / Linux

Windows desktop console for llama.cpp runtimes, models, and local coding workflows

Pre-compiled Python whl for Flash-attention, SageAttention, NATTEN, xFormer etc