inference-sim/inference-sim

Accurate, large-scale, and extensible simulator for LLM inference Systems

AISimulate predicts LLM serving behavior and searches for strong deployment configurations offline, without bringing up a GPU serving cluster

A Lightweight LLM Inference Performance Simulator

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

LLMServingSim 2.0: A Unified Simulator for Heterogeneous and Disaggregated LLM Serving Infrastructure

Frontier: A Discrete-Event Simulator for Modern LLM Serving

A lightweight, configurable, and real-time simulator designed to mimic the behavior of vLLM without the need for GPUs or running actual heavy models.

Offline optimization of your disaggregated Dynamo graph

Benchmark and optimize LLM inference across frameworks with ease