AI-Efficiency/BiBERT

Official implementation of LittleBit (NeurIPS 2025) and its follow-up LittleBit-2 (ICML 2026)

🛠️ Tools for Transformers compression using PyTorch Lightning ⚡

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Official Pytorch repository for Extreme Compression of Large Language Models via Additive Quantization https://arxiv.org/pdf/2401.06118.pdf and PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression https://arxiv.org/abs/2405.14852

For releasing code related to compression methods for transformers, accompanying our publications

0️⃣1️⃣🤗 BitNet-Transformers: Huggingface Transformers Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch with Llama(2) Architecture

The official implementation of the paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".

Bringing BERT into modernity via both architecture changes and scaling

An Efficient Compression Framework for LLM