jingyaogong/minimind

This project aims to replicate mainstream open-source model architectures with limited computational resources, implementing mini models with 100-200M parameters.

"LLM from Zero to Hero: An End-to-End Large Language Model Journey from Data to Application!"

A straightforward method for training your LLM, from downloading data to generating text.

Build LLM from scratch

A 124.6M LLM trained from scratch on 13B tokens on a single RTX 4090 — tokenizer, pretraining, post-training, evaluation, and reproducible inference.

LLM Tutorial for Everyone.

Build a modern LLM from scratch. Every line commented. Explained like we are five.

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

The fastest way to build and start training your own LLM. CLI tool that scaffolds production-ready PyTorch training projects in seconds. Like create-next-app but for language models.