aws-neuron/neuronx-distributed-inference

Community maintained hardware plugin for vLLM on AWS Neuron

Training and inference on AWS Trainium and Inferentia chips.

Powering AWS purpose-built machine learning chips. Blazing fast and cost effective, natively integrated into PyTorch and TensorFlow and integrated with your favorite AWS services

Yet another minimalist deep-learning framework optimized for inference

A native PyTorch backend for AWS Trainium with support for eager execution, torch.compile, distributed training, and custom NKI kernels

One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF block formats, ternary Bonsai 2, device route profiles, MTP and DFlash2.