Supercharge Your Model Training
Like mosaicml/composer, 0 likes 0 Bookmark mosaicml/composer, 0 bookmarks 0 A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
Like NVIDIA/apex, 0 likes 0 Bookmark NVIDIA/apex, 0 bookmarks 0 🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
Like huggingface/accelerate, 0 likes 0 Bookmark huggingface/accelerate, 0 bookmarks 0 Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.
Like Lightning-AI/pytorch-lightning, 0 likes 0 Bookmark Lightning-AI/pytorch-lightning, 0 bookmarks 0 PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.
Like Lightning-AI/lightning-thunder, 0 likes 0 Bookmark Lightning-AI/lightning-thunder, 0 bookmarks 0 DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Like deepspeedai/DeepSpeed, 0 likes 0 Bookmark deepspeedai/DeepSpeed, 0 bookmarks 0 Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)
Like meta-pytorch/torchft, 0 likes 0 Bookmark meta-pytorch/torchft, 0 bookmarks 0 Making large AI models cheaper, faster and more accessible
Like hpcaitech/ColossalAI, 0 likes 0 Bookmark hpcaitech/ColossalAI, 0 bookmarks 0