keyu-wang-2002/DepthBench

Official Project Page for Deep Delta Learning (https://arxiv.org/abs/2601.00417)

A comprehensive, cutting-edge open-source framework for next-generation Transformers, with compute-matched benchmarking and a platform for autonomous architecture research.

A concise but complete full-attention transformer with a set of promising experimental features from various papers

A place to store reusable transformer components of my own creation or found on the interwebs

Hackable and optimized Transformers building blocks, supporting a composable construction.

DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation

A collection of tricks and tools to speed up transformer models