PrimeIntellect-ai/OpenDiloco

Distributed Training Over-The-Internet

Making large AI models cheaper, faster and more accessible

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Minimalistic 4D-parallelism distributed training framework for education purpose

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Minimalistic large language model 3D-parallelism training

Ongoing research training transformer models at scale

Fault-tolerant, highly scalable GPU orchestration, and a machine learning framework designed for training models with billions to trillions of parameters

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support