melisa-writer/short-transformers

The official implementation of the paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡

A collection of tricks and tools to speed up transformer models

✂️Scalpel: Recovery-Aware Layer Pruning for Faster Vision-Language Models

For releasing code related to compression methods for transformers, accompanying our publications

Export utility for unconstrained channel pruned models

This is our own implementation of 'Layer Selective Rank Reduction'

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

A place to store reusable transformer components of my own creation or found on the interwebs