vllm-project/guidellm

Open Source Inference Research Platform Standard / 开源推理研究平台

A high-throughput and memory-efficient inference and serving engine for LLMs

Lighteval is your all-in-one toolkit for evaluating LLMs across multiple backends

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

🧊 Open source LLM observability platform. One line of code to monitor, evaluate, and experiment. YC W23 🍓

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Evaluation and Tracking for LLM Experiments and AI Agents

Optimizing inference proxy for LLMs

Large-scale LLM inference engine