MollySophia/rwkv-qualcomm

Inference RWKV with multiple supported backends.

INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model

Fully open-source, high-performance LLM inference engine for the RK3588 NPU

RWKV-7 7.2B fp16 15000+ tps decoding @ single 5090

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

Code sample showing how to run and benchmark models on Qualcomm's Window PCs

onnxruntime-qnn is the Qualcomm AI Runtime (QAIRT) execution provider for onnxruntime. It provides onnxruntime hardware acceleration and advanced functionalities on Qualcomm devices.

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

Qualcomm Cloud AI SDK (Platform and Apps) enable high performance deep learning inference on Qualcomm Cloud AI platforms delivering high throughput and low latency across Computer Vision, Object Detection, Natural Language Processing and Generative AI models.