NX-AI/xlstm

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

Mamba SSM architecture

🚀 Efficient implementations for emerging model architectures

A concise but complete full-attention transformer with a set of promising experimental features from various papers

PyTorch and TensorFlow implementation of NCP, LTC, and CfC wired neural models

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

A minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training

Implement a ChatGPT-like LLM in PyTorch from scratch, step by step

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch