huggingface/nanotron

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Train transformer language models with reinforcement learning.

Muon is Scalable for LLM Training

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

A simple, performant, and scalable Jax LLM!

Go ahead and axolotl questions

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

A Lightweight LLM Post-Training Library

Ongoing research training transformer models at scale