VectorInstitute/vector-inference

sparkrun - launch, manage, and stop LLM inference workloads on NVIDIA DGX Spark systems. Live support: https://discord.com/invite/GH5kRgv6ZD

Optimized inference for the DGX Spark and spark clusters

QuantumFlow - Distributed LLM inference scheduling framework with multi-backend support (vLLM, TGI, SGLang), adaptive scheduling strategies, and cluster management.

Achieve state of the art inference performance with modern accelerators on Kubernetes

Scale LLM Engine public repository

Evidence-driven SGLang inference deployment optimization.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

llm-d benchmark scripts and tooling

A flexible serving framework that delivers efficient and fault-tolerant LLM inference for clustered deployments.