Blackwellboy/model-serving-minefield

LLM evals to test open-source inference providers

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Tool-calling quality benchmark for LLM serving stacks. 80+ deterministic scenarios testing multi-turn orchestration, safety boundaries, and structured output. Supports vLLM, SGLang, and llama.cpp.

Comprehensive LLM Error Analysis and Reporting

Model authenticity / dilution detection CLI: one command, one report. Detect fake or watered-down LLM endpoints via identity, context, tokenizer, capability and latency checks.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

Find the data that broke your fine-tune.

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

AI Observability & Evaluation