sqliteai/warp

Run models larger than your RAM or VRAM by streaming MoE experts from SSD. Built on llama.cpp.

High-speed Large Language Model Serving for Local Deployment

A custom inference engine tuned for Qwen4 architecture. Currently runs Qwen3.8-Flash-Next at 15+ tok/s on 16GB of VRAM and 30 GB of RAM

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand

Runs 405B LLMs on 8GB VRAM

Memory-aware GGUF inference runtime with OpenAI-compatible serving

AirLLM 2.8T LLM inference with single 4GB GPU

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware