guqiong96/Lvllm

LLM inference in C/C++

A highly optimized LLM inference acceleration engine for Llama and its variants.

~1300 line, minimal, extensible LLM inference engine built from scratch.

A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official (NV)FP4 checkpoint's quality on consumer Blackwell cards

A high-throughput and memory-efficient inference and serving engine for LLMs

FlashInfer: Kernel Library for LLM Serving

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

V100 / SM70-focused vLLM engineering fork for modern LLM inference.

Pure Rust Inference Engine