microsoft/Olive

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

Pruna is a model optimization framework built for developers, enabling you to deliver faster, more efficient models with minimal overhead.

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

Simplify your onnx model

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

A retargetable MLIR-based machine learning compiler and runtime toolkit.