AutoGPTQ/AutoGPTQ

Accessible large language models via k-bit quantization for PyTorch.

State-of-the-art LLM compression, built for production inference with vLLM

🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

Utilities intended for use with Llama models.

20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

AirLLM 70B inference with single 4GB GPU

A fast inference library for running LLMs locally on modern consumer-class GPUs

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.