stevibe/ToolCall-15

Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, and structured output. Supports vLLM, SGLang, and llama.cpp.

[ICLR 2026] The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution

MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers

A benchmark to evaluate language models on questions I've previously asked them to solve.

[ICLR'24 spotlight] An open platform for training, serving, and evaluating large language model for tool learning.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

bloom - evaluate any behavior immediately  🌸🌱

Measuring and evolving with the frontier of agent work