Wenyueh/MinivLLM

Nano vLLM

~1300 line, minimal, extensible LLM inference engine built from scratch.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Pure Rust Inference Engine

FlashInfer: Kernel Library for LLM Serving

Experimental llama.cpp fork for inference research and development

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

A high-throughput and memory-efficient inference and serving engine for LLMs

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉