Beomi/BitNet-Transformers

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Official inference framework for 1-bit LLMs

Lightweight toolkit package to train and fine-tune 1.58bit Language models

Accessible large language models via k-bit quantization for PyTorch.

Implementing LLM architectures from scratch in PyTorch

PyTorch implementations of modern open-source LLM architectures (Llama, Qwen, DeepSeek, Gemma, GPT-OSS, Kimi, and more) — written from scratch for readability and learning, based on Sebastian Raschka's LLM Architecture Gallery.

The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Official implementation of LittleBit (NeurIPS 2025) and its follow-up LittleBit-2 (ICML 2026)

Build LLM from scratch