Lightning-AI/lightning-thunder

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

PyTorch native quantization for training and inference

You like pytorch? You like micrograd? You love tinygrad! ❤️

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Supercharge Your Model Training

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch