leoheuler/flashtensors

FlashInfer: Kernel Library for LLM Serving

Experimental llama.cpp fork for inference research and development

~1300 line, minimal, extensible LLM inference engine built from scratch.

Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.

Based on Nano-vLLM, a simple replication of vLLM with self-contained paged attention and flash attention implementation

High-speed Large Language Model Serving for Local Deployment