lechmazur/sycophancy

Auditing LLM-as-judge systems with paired synthetic controls, bootstrap CIs, and BH-FDR bias detection.

Adversarial multi-turn benchmark for LLM debate quality, using side-swapped matchups and multi-model judging to rank models by judged debate performance.

BullshitBench measures whether AI models challenge nonsensical prompts instead of confidently answering them, created by Peter Gostev.

Simple LLM Evaluation Using LLM As a Judge 👩‍⚖️

Code and data for the paper: On the Humanity of Conversational AI: Evaluating the Psychological Portrayal of LLMs

LangFair is a Python library for conducting use-case level LLM bias and fairness assessments

bloom - evaluate any behavior immediately  🌸🌱

One-shot LLM eval cases by NAGI STUDIO - same prompt, different agents (model + harness), runnable artifacts side by side.