AnswerDotAI/fsdp_qlora

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Single-GPU LLM fine-tuning preflight: memory estimates, runnable LoRA/QLoRA recipes, and local measurements.

Go ahead and axolotl questions

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

A simple, performant, and scalable Jax LLM!

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

Accessible large language models via k-bit quantization for PyTorch.

Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization

🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.