SamsungLabs/LittleBit

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

Python package for LLM compression

State-of-the-art LLM compression, built for production inference with vLLM

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

Awesome LLM compression research papers and tools.

Accessible large language models via k-bit quantization for PyTorch.

For releasing code related to compression methods for transformers, accompanying our publications