fkodom/transformer-from-scratch

Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

哈佛大学 Transformer 经典入门教程 annotated-transformer-Chinese 中文版 Transformer 论文 Attention is All You Need 的 pytorch 中文注释代码实现，翻译自harvardnlp/annotated-transformer

Build LLM from scratch

A concise but complete full-attention transformer with a set of promising experimental features from various papers

Sutskever 30 implementations inspired by https://papercode.vercel.app/ | For Agents, use https://github.com/pageman/Sutskever-Agent | Polyglot / Multi-Backed version at https://github.com/pageman/sutskever-30-beyond-numpy

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.

Implementation of the conditionally routed attention in the CoLT5 architecture, in Pytorch