flexflow/flexflow-train

DLRover: An Automatic Distributed Deep Learning System

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

Making large AI models cheaper, faster and more accessible

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Supercharge Your Model Training

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch