curvedinf/voodoo-dyn-quant

Produce your own Dynamic 3.0 Quants and achieve optimum accuracy & SOTA quantization performance! Input a target size and the toolchain will create a GGUF recipe tuned to your hardware within seconds — flexible model sizing and lowest achievable perplexity/kld for GGUF enthusiasts seeking precise and automated dynamic quant production.

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

Adaptive Precision for EXpert Models: MoE-aware mixed-precision quantization

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

An Efficient Compression Framework for LLM

State-of-the-art LLM compression, built for production inference with vLLM

Python package for LLM compression

Accessible large language models via k-bit quantization for PyTorch.

Deprecated: now LiteRT Quantizer