OpenMachine-ai/transformer-tricks

Hackable and optimized Transformers building blocks, supporting a composable construction.

Utils for Unsloth https://github.com/unslothai/unsloth

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

For releasing code related to compression methods for transformers, accompanying our publications

The official implementation of the paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".

A concise but complete full-attention transformer with a set of promising experimental features from various papers

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

Accelerate your Hugging Face Transformers 7.6-9x. Native to Hugging Face and PyTorch.

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.