MuLabPKU/TransArch

Awesome LLM compression research papers and tools.

ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization

Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

VT-Transformer — a 1x1x1 LLM inference engine built on AI-generated kernels.

📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

Turning LLM architecture exploration into reusable knowledge for the community. 🌳

Based on Nano-vLLM, a simple replication of vLLM with self-contained paged attention and flash attention implementation

📚A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.🎉