Helsinki-NLP/Tatoeba-Challenge

Repo for the Belebele dataset, a massively multilingual reading comprehension dataset.

We introduce MKQA, an open-domain question answering evaluation set comprising 10k question-answer pairs aligned across 26 typologically diverse languages (260k question-answer pairs in total). The goal of this dataset is to provide a challenging benchmark for question answering quality across a wide set of languages. Please refer to our paper for details, MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering

A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB team.

State-of-the-art LLM-based translation models.

A list of awesome Machine Translation frameworks, libraries, software and papers

The robust European language model benchmark.

darija <-> english dataset

⚡LLM Zoo is a project that provides data, models, and evaluation benchmark for large language models.⚡