Ajay9o9/runs-on-a-3060

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.

Tools, ComfyUI workflows and benchmark configs from a 4x RTX 3090 local-inference rig

Practical local LLM recipes and benchmarks for RTX 5060 Ti setups

Memory Management for the GPU Poor, run the latest open source frontier models on consumer Nvidia GPUs

Local AI registry: one validated recipe per machine, with the evidence attached

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

TensorRT Extension for Stable Diffusion Web UI

Find the best models and how to run them locally.

A fast AI Video Generator for the GPU Poor. Supports Minimax H3, LTX-2.3/2.5, Wan 2.1/2.2, Qwen Image 1/2, Krea 2, Hunyuan Video, LTX Video and Flux.