moonshine-ai/qc_npu_benchmark

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

QAI AppBuilder is designed to help developers easily execute models on WoS and Linux platforms. It encapsulates the Qualcomm® AI Runtime SDK APIs into a set of simplified interfaces for running models on the NPU/HTP.

MLPerf Client is a benchmark for Windows, Linux and macOS, focusing on client form factors in ML inference scenarios.

Examples for using ONNX Runtime for machine learning inferencing.

The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

Peak compute, memory bandwidth and latency of CPUs, GPUs and NPUs: the same tests through every API that reaches a device. Command line and app for Windows, macOS, Linux and Android

High-performance single-GPU inference for selected model checkpoints and GPUs.