runpod-workers/worker-vllm

Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.

A high-throughput and memory-efficient inference and serving engine for LLMs

Large-scale LLM inference engine

The official API server for Exllama. OAI compatible, lightweight, and fast.

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

Universal LLM Deployment Engine with ML Compilation

Large Language Model Text Generation Inference

Scale LLM Engine public repository

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.