lateos-ai/reflex

Fast local LLM inference for the GPU you already have: one Rust binary, GGUF models, an OpenAI-compatible server and a chat page with a ghost.

High-speed Large Language Model Serving for Local Deployment

Pure Rust Inference Engine

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

Memory-aware GGUF inference runtime with OpenAI-compatible serving

High-performance single-GPU inference for selected model checkpoints and GPUs.

One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF block formats, ternary Bonsai 2, device route profiles, MTP and DFlash2.

CUDA Blackwell performance fork of antirez/ds4 (DeepSeek-V4-Flash): ~4x prefill, ~1.5x decode, continuous batching, weight server, DSpark

Experimental llama.cpp fork for inference research and development