Picovoice/picollm

High-speed Large Language Model Serving for Local Deployment

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

Run a 1-billion parameter LLM on a $10 board with 256MB RAM

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

A fast inference library for running LLMs locally on modern consumer-class GPUs

Generative AI with ONNX Runtime