nopnoping/micro-vllm

Based on Nano-vLLM, a simple replication of vLLM with self-contained paged attention and flash attention implementation

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Nano vLLM

Learn LLM Inference Engineering step by step - from KV cache, PagedAttention, and continuous batching to vLLM, SGLang, and GPUs.

everything you need to know about inference engineering

~1300 line, minimal, extensible LLM inference engine built from scratch.

llm-inference

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

A vLLM-inspired inference engine for looped transformers, with loop-level scheduling and execution across composable stages