containers/ramalama

Docker Model Runner

Run any agent on any model, models stored as OCI images

Local AI API Platform

Stop configuring your AI stack. Start using it. One command brings a complete pre-wired LLM stack with hundreds of services to explore.

A high-performance inference engine for AI models

Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.