hsperus/minnak-gpt

Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.

Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch

Sutskever 30 implementations inspired by https://papercode.vercel.app/ | For Agents, use https://github.com/pageman/Sutskever-Agent | Polyglot / Multi-Backed version at https://github.com/pageman/sutskever-30-beyond-numpy

Repo for ML Models built from scratch such as Self-Attention, Linear +Logistic Regression, PCA, LDA. CNN, LSTM, Neural Networks using Numpy only

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

A deep learning library built from scratch with complex neural networks examples built on top for learning purposes.

Pretraining Transformers Without Backpropagation

Transformer architecture explained step by step - the full architecture, every attention variant, positional embeddings, and every layer inside a Transformer.