triton-inference-server/triton_cli

The simplest way to serve AI/ML models in production

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Universal LLM Deployment Engine with ML Compilation

Scale LLM Engine public repository

A simple TUI for serving local LLM models. Pick a model, pick a backend, serve it

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

Common source, scripts and utilities for creating Triton backends.

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.