NVIDIA/nccl-extensions

MSCCL++: A GPU-driven communication stack for scalable AI applications

DeepEP: an efficient expert-parallel communication library

MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

Dist-MoE implements Speed-of-Light distributed Mixture-of-Experts forward and backward kernels. They are highly optimized and fused comm-compute kernels for training and inference on SM100+ GPUs

torchcomms: a modern PyTorch communications API

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

Optimized primitives for collective multi-GPU communication

A high-performance communication library for machine learning training and inference on Huawei Ascend NPUs.