patchy631/time-to-first-token

100 days of LLM inference engineering — daily posts, experiments, and visualizations

Everything you need to know about LLM inference

learn LLM inference system on Apple Silicon for systems engineers: build a tiny vLLM + Qwen

LLM notes, including model inference, transformer model structure, and llm framework code analysis notes.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Official SGLang x Datawhale course on LLM inference: understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. Available in English and Chinese.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

TokenSpeed is a speed-of-light LLM inference engine.

Efficient AI Inference & Serving