huggingface/ratchet

a fast cross platform AI inference engine 🤖 using Rust 🦀 and WebGPU 🎮

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

High-performance In-browser LLM Inference Engine

Run local LLMs like llama, deepseek-distill, kokoro and more inside your browser

Universal LLM Deployment Engine with ML Compilation

WebAssembly binding for llama.cpp - Enabling on-browser LLM inference

Production ready toolkit to run AI locally

LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization

High-performance LLM, VLM, embedding, reranking, and audio inference for Apple Silicon and NVIDIA CUDA systems.