IST-DASLab/SLQ

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Gumbel-Softmax post-training quantization for LLMs (1–3 bit scalar, INT/GGUF-compatible).

[ICML 2024 Oral] This project is the official implementation of our Accurate LoRA-Finetuning Quantization of LLMs via Information Retention

An Efficient Compression Framework for LLM

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

LLM quantization optimizer

Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization

AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:

Making 1.58-bit models simple to build and broadly accessible through low-cost, high-accuracy post-training ternary quantization.