NVIDIA/apex

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

The fastai deep learning library

You like pytorch? You like micrograd? You love tinygrad! ❤️

Tensors and Dynamic neural networks in Python with strong GPU acceleration

Accessible large language models via k-bit quantization for PyTorch.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries