Xilinx/brevitas

PyTorch native quantization for training and inference

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Accessible large language models via k-bit quantization for PyTorch.

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Vector (and Scalar) Quantization, in Pytorch

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

A simple and effective post training quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的后训练量化工具包