tile-ai/TileOPs

A kernel library written in tilelang

High Performance LLM Inference Operator Library

DeepGEMM: clean and efficient BLAS kernel library on GPU

Mojo Opset is a collection of different high-performance kernel implementations for LLM and multimodal.

FlagGems is an operator library for large language models implemented in the Triton Language.

FlashInfer: Kernel Library for LLM Serving

Optimized GPU compiler for LLM inference. Choose from a list of optimized recipes or optimize your own model via kernel fusion, autotuning, and advanced scheduling. Run benchmarks across different GPU types and configurations, track results and share experiments with the community.

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

FlagGems-sglang is part of FlagOS. It is a high-performance operator library designed for multiple hardware backends.