microsoft/dion

For optimization algorithm research and development.

Schedule-Free Optimization in PyTorch

Muon is an optimizer for hidden layers in neural networks

Distributed Training Over-The-Internet

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

Supercharge Your Model Training

PyTorch native quantization for training and inference

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools