InternLM/xtuner

Compact and Agent-Native MoE Training System

Minimalistic large language model 3D-parallelism training

Muon is Scalable for LLM Training

LM engine is a library for pretraining/finetuning LLMs

XoRL

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

MCore-Bridge: Providing Megatron-Core model definitions for state-of-the-art large models and making Megatron training as simple as Transformers — with support for 300+ large language models (Qwen3-Next, GLM-5.3, Deepseek-V4, MiniMax-2.7, ...) and 200+ multimodal large models (Qwen3.8, Qwen3-Omni, Gemma4, ...).

Modalities, a PyTorch-native framework for distributed and reproducible foundation model training.