CASE-Lab-UMD/LLM-Drop

This is our own implementation of 'Layer Selective Rank Reduction'

For releasing code related to compression methods for transformers, accompanying our publications

A collection of tricks and tools to speed up transformer models

[EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.

State-of-the-art LLM compression, built for production inference with vLLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

Awesome LLM compression research papers and tools.

Python package for LLM compression

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.