databricks/flashoptim

Gefen: Optimized Stochastic Optimizer

A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch

Efficient optimizers

PyTorch native quantization for training and inference

Flash-Muon: An Efficient Implementation of Muon Optimizer

An efficient implementation of learned optimizers in PyTorch

FlexTensor is a tensor offloading and management library for PyTorch that enables running large models on limited GPU memory by intelligently offloading tensors between GPU and CPU memory.

A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit and 4-bit floating point (FP8 and FP4) precision on Hopper, Ada and Blackwell GPUs, to provide better performance with lower memory utilization in both training and inference.