forecastingresearch/forecastbench

Anyone can predict the future. Almost nobody keeps the receipt. A hash-chained prediction ledger for LLMs — sealed before the answer exists, graded with proper scoring rules.

"FutureShow: Can AI Predict the Future? Live Real-World Forecasting"

Evaluating LLMs with Dynamic Data

TimeCopilot: the GenAI Forecasting Agent. Built on LLMs and Time Series Foundation Models, it lets you forecast, cross-validate, and detect anomalies using multiple foundation models through a single API. From finance and energy to web analytics, TimeCopilot turns natural-language queries into production-ready forecasts.

Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

LiveBench: A Challenging, Contamination-Free LLM Benchmark

Forecast evaluation library