huggingface/optimum-quanto

PyTorch native quantization for training and inference

Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization

A simple and effective post training quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的后训练量化工具包

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Brevitas: neural network quantization in PyTorch

Accessible large language models via k-bit quantization for PyTorch.

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.