crabml/crabml

AI inference, packed simply. A blazing-fast, zero-dependency WebGPU runtime to run GGUF models directly in the browser. Features a symmetric API for seamless local execution and cloud provider routing. Built with Rust & C++.

Open Source Brain Platform | AI training and inference in Rust

Blazing-fast LLM inference in pure Rust. No PyTorch and Python runtime.

Pure-Rust GGUF inference engine with quantized CPU, Metal & CUDA kernels, MoE support, OpenAI-compatible server. Benchmarked head-to-head against llama.cpp.

A cross-platform browser ML framework.

High-performance In-browser LLM Inference Engine

Fast, flexible LLM inference

A Pure Rust based LLM, VLM, VLA, TTS, OCR Inference Engine, powering by Candle & Rust. Alternate to your llama.cpp but much more simpler and cleaner..

WebAssembly binding for llama.cpp - Enabling on-browser LLM inference