melonmath/microkimi

A minimal MiniCPM5-1B model inference engine built purely in Rust

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime.

A lightweight vLLM-style inference engine in Rust

[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool agents, quantization-aware training. Scales: Tiny (25M) to Large (1.3B).

CPU-only, pure-Rust implementation of LiquidAI's LFM2.5-8B-A1B LLM

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

fastllm是后端无依赖的高性能大模型推理库，帮助您在大部分设备上高效推理大模型！