PrunaAI/pruna

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

Olive: Simplify ML Model Finetuning, Conversion, Quantization, and Optimization for CPUs, GPUs and NPUs.

PyTorch native quantization for training and inference

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.