cloudrift-ai/emmy

Benchmark and optimize LLM inference across frameworks with ease

Evidence-driven SGLang inference deployment optimization.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

FlashInfer: Kernel Library for LLM Serving

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Automating llama.cpp LLM parameter optimization for highest speed/best fit using Morris/Taguchi DoE sweeps.

Offline optimization of your disaggregated Dynamo graph

Inferflow is an efficient and highly configurable inference engine for large language models (LLMs).