Tylogi/TyloQuant

High-speed Large Language Model Serving for Local Deployment

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

On-device LLM Inference Powered by X-Bit Quantization

llama.cpp fork with TurboQuant WHT-rotated KV cache & weight compression + Gemma 4 MTP and Qwen 3.6 NextN speculative decoding (+30-50% throughput).

FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.

A fast inference library for running LLMs locally on modern consumer-class GPUs

A custom inference engine tuned for Qwen4 architecture. Currently runs Qwen3.8-Flash-Next at 15+ tok/s on 16GB of VRAM and 30 GB of RAM