raymin0223/mixture_of_recursions

Experiments around a simple idea for inducing multiple hierarchical predictive model within a GPT

Official Project Page for Deep Delta Learning (https://arxiv.org/abs/2601.00417)

Official Project Page for Recurrent Looped Transformer (RLT) (https://arxiv.org/abs/2610.07591)

RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

A theoretical reconstruction of the Claude Mythos architecture, built from first principles using the available research literature.

Mamba SSM architecture

[NeurIPS 2025 Spotlight] TPA: Tensor ProducT ATTenTion Transformer (https://arxiv.org/abs/2501.06425)

The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction