lvyufeng/PocketLLM

High-speed Large Language Model Serving for Local Deployment

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

QVAC Fabric: cross-platform LLM inference and fine-tuning, optimized for edge devices and heterogenous GPUs

Runs 405B LLMs on 8GB VRAM

Run a 1-billion parameter LLM on a $10 board with 256MB RAM

A fast inference library for running LLMs locally on modern consumer-class GPUs

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

AirLLM 2.8T LLM inference with single 4GB GPU