AtomicBot-ai/atomic-llama-cpp-turboquant

Experimental llama.cpp fork for inference research and development

LLM inference in C/C++

llama.cpp fork with additional SOTA quants and improved performance

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

LLM inference in C/C++

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

A fast inference library for running LLMs locally on modern consumer-class GPUs

LLM inference in C/C++

High-speed Large Language Model Serving for Local Deployment