NVIDIA/nccl-tests

Optimized primitives for collective multi-GPU communication

KernelBench: Can LLMs Write GPU Kernels? - Benchmark + Toolkit with Torch -> CUDA (+ more DSLs)

Multi-GPU CUDA stress test

Build and run containers leveraging NVIDIA GPUs

An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

Reference implementations of MLPerf® inference benchmarks

Collective communications library with various primitives for multi-machine training.

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

GPGPU-Sim provides a detailed simulation model of contemporary NVIDIA GPUs running CUDA and/or OpenCL workloads. It includes support for features such as TensorCores and CUDA Dynamic Parallelism as well as a performance visualization tool, AerialVisoin, and an integrated energy model, GPUWattch.