Modalities/modalities

Minimalistic large language model 3D-parallelism training

Go ahead and axolotl questions

A lightweight, AI-native training framework for large language models. Designed for fast iteration, reproducible experiments, and modular configuration across SFT, RLVR, and evaluation workflows.

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

Muon is Scalable for LLM Training

A simple, performant, and scalable Jax LLM!

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support