FasterAI-Labs/fasterai

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

PyTorch native quantization for training and inference

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

Neural Network Compression Framework for enhanced OpenVINO™ inference

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

Pruna is a model optimization framework built for developers, enabling you to deliver faster, more efficient models with minimal overhead.

Model Compression Toolkit (MCT) is an open source project for neural network model optimization under efficient, constrained hardware. This project provides researchers, developers, and engineers advanced quantization and compression tools for deploying state-of-the-art neural networks.

Fast & Simple Resource-Constrained Learning of Deep Network Structure

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT