tensorflow/model-optimization

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Pruna is a model optimization framework built for developers, enabling you to deliver faster, more efficient models with minimal overhead.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.

PyTorch native quantization for training and inference

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

Simplify your onnx model