denoflore/greenboost-windows

Native Windows vLLM: 0.27.1 Latest; 0.29.0 Python 3.14/CUDA 13.2 release (CPU TorchAudio), plus Python 3.13/CUDA 13.0 prerelease. PyTorch 2.13, FlashAttention/Rust, OpenAI serving, 10 KV formats, Multi-TurboQuant, experimental CPU/NVMe KV offload. No WSL or Docker.

An htop for what's inside your NVIDIA GPU on Windows: process memory, engine activity, and LLM weights/KV-cache introspection

VRAMDISK: Create VRAM disk on Windows

Use your NVIDIA GPU's VRAM as swap space on Linux. Built for laptops with soldered memory and no upgrade path. If you have an RTX card sitting there with 8GB of VRAM and you're getting swapped to SSD, this puts that VRAM to work

Run CUDA-targeted Windows applications on AMD GPUs with ZLUDA + ROCm/HIP.

Dynamic Memory Management for Serving LLMs without PagedAttention

LLM inference in C/C++

Ollama-compatible LLM server on a tuned llama.cpp fork. Beats vllm by 40% on single stream decode, and by 20% on multi stream decode, and beats ollama by much 2-4x. Made possible with custom CUDA kernels, MTP, speculation and its own 3-bit builds. Uncensors models in one command. Windows and Linux, NVIDIA. Optimized to not eat up ram.

Delta Inference Engine: DeepSeek-V4.1-Flash on one NVIDIA GPU, natively on Windows (OpenAI-compatible server)