MERA-Evaluation/MERA

The robust European language model benchmark.

The code and data for "MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark" [NeurIPS 2024]

ParsBench provides toolkits for benchmarking LLMs based on the Persian language tasks.

Repo for the Belebele dataset, a massively multilingual reading comprehension dataset.

⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡

We introduce MKQA, an open-domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). The goal of this dataset is to provide a challenging benchmark for question answering quality across a wide set of languages. Please refer to our paper for details, MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering

MTEB: State-of-the-art evaluation of embeddings across languages and modalities