modular/llm-inference-handbook

100 days of LLM inference engineering — daily posts, experiments, and visualizations

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

Optimizing inference proxy for LLMs

Scale LLM Engine public repository

Open Source Inference Research Platform Standard / 开源推理研究平台

FlashInfer: Kernel Library for LLM Serving

A high-throughput and memory-efficient inference and serving engine for LLMs