vipshop/cache-dit

xDiT: A Scalable Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism

https://wavespeed.ai/ Best inference performance optimization framework for HuggingFace Diffusers on NVIDIA GPUs.

DFlash: Block Diffusion for Flash Speculative Decoding

TokenSpeed is a speed-of-light LLM inference engine.

TensorRT Extension for Stable Diffusion Web UI

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

PyTorch compiler that accelerates training and inference. Get built-in optimizations for performance, memory, parallelism, and easily write your own.

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.

High-performance single-GPU inference for selected model checkpoints and GPUs.