nntrainer/nntrainer

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

On-device AI across mobile, embedded and edge for PyTorch

LiteRT-LM is Google's production-ready, high-performance, open-source inference framework for deploying Large Language Models on edge devices.

QVAC Fabric: cross-platform LLM inference and fine-tuning, optimized for edge devices and heterogenous GPUs

LiteRT, successor to TensorFlow Lite. is Google's On-device framework for high-performance ML & GenAI deployment on edge platforms, via efficient conversion, runtime, and optimization

On-device LLM Inference Powered by X-Bit Quantization

Smart inference engine that adapts to your hardware to run models faster, even when they don’t fit in memory

High-performance, light-weight C++ LLM and VLM Inference Software for Physical AI

Awesome Mobile LLMs