pykeio/ort

Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK. Python, C# and JS(WASM) bindings available.

Inference at the speed of light.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

Examples for using ONNX Runtime for machine learning inferencing.

Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference

ONNX-TensorRT: TensorRT backend for ONNX

Burn is a next generation tensor library and Deep Learning Framework that doesn't compromise on flexibility, efficiency and portability.

Minimalist ML framework for Rust

Instant, controllable, local pre-trained AI models in Rust