pytorch/FBGEMM

Pytorch domain library for recommendation systems

DeepGEMM: clean and efficient BLAS kernel library on GPU

FlashInfer: Kernel Library for LLM Serving

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

PyTorch native quantization for training and inference

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

A fast communication-overlapping library for tensor/expert parallelism on GPUs.

Transformer related optimization, including BERT, GPT

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT