surya-koritala/Glyd

A Lossless Compression Library for AI pipelines

For releasing code related to compression methods for transformers, accompanying our publications

Memory Management for the GPU Poor, run the latest open source frontier models on consumer Nvidia GPUs

A novel take on lossless data compression

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

AI Model Demand Offloading Allocator

AirLLM 2.8T LLM inference with single 4GB GPU

LLM KV cache compression made easy