anuj0456/OpenArch

Implementing LLM architectures from scratch in PyTorch

Turning LLM architecture exploration into reusable knowledge for the community. 🌳

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Code files for course "A deep understanding of AI large language model mechanisms"

Build LLM from scratch

Under the Hood - Build Every Layer of a Large Language Model from Scratch. A 36-project, build-it/break-it/measure-it manual by Ramchand Kumaresan.

Readable, composable PyTorch library for language models: build Llama, Qwen, Gemma, DeepSeek, Kimi and 20+ other architectures from plain nn.Modules, then train them on CPU, one GPU, or multi-GPU.

A hands-on course on modern LLM architectures, training, and inference, with step-by-step PyTorch implementations and runnable notebooks.