openvinotoolkit/nncf

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

Pruna is a model optimization framework built for developers, enabling you to deliver faster, more efficient models with minimal overhead.

A Toolkit to Help Optimize Onnx Model

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

PyTorch native quantization for training and inference

Fast & Simple Resource-Constrained Learning of Deep Network Structure