Phi-Bench (Φ-Bench): 85 open-source LLM-infrastructure engineering tasks for frontier LLMs & coding agents (KFC/LH/E2E). Self-contained public Dockerfile + offline scoring + reference solution per task. Website: llminfrabench.com
Run the AEON Bench suite on your own hardware: verified HuggingFace pull → serve → benchmark (text · agentic ×3 harnesses · vision · audio · arena · perf) → ed25519-signed attested submit.
Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
Local-first CLI for benchmarking LLMs on real hardware — quality, speed, reliability, and a real multi-turn agent loop.
Benchmarking Open-Ended Inference Optimization by AI Agents
CloudAI Benchmark Framework
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
A validation and profiling tool for AI infrastructure
Collection of evals for Inspect AI
