prabanta-dev/janas

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

[Official] Prima.cpp: Scale Your Local AI Beyond One Device.

Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.

Qwen3.8-Flash-Next on any consumer hardware: one-click install for Windows / Linux. Strata inference engine, OpenAI/Anthropic API on localhost, optional image input.

A fork of https://github.com/Niko1221/Strata optimized for my 3090+5070 Ti setup, see docs/COMPARISON.md. Renamed to Lamina.

High-speed Large Language Model Serving for Local Deployment

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

LLM inference in C/C++

llama.cpp fork with additional SOTA quants and improved performance