RaymondHuang210129/llama.cpp-adaptive-kv-streaming

KVMem virtualizes KV memory across inference backends to make ultra-long-context local AI accessible on limited hardware.

Experimental llama.cpp fork for inference research and development

KVarN, KV cache precision tail, low-bit quants in llama.cpp for longer context of better precision in the same VRAM

Dynamic Memory Management for Serving LLMs without PagedAttention

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

TurboQuant: Near-optimal KV cache quantization for LLM inference (3-bit keys, 2-bit values) with Triton kernels + vLLM integration

LLM inference in C/C++

LLM inference in C/C++

llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).