jungledesh/profile

Benchmark and optimize LLM inference across frameworks with ease

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

Evidence-driven SGLang inference deployment optimization.

Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs

Real-time monitoring for llama.cpp inference servers: a read-only host agent plus a web dashboard for hardware, health and inference telemetry (prefill/decode vs context depth, slot occupancy, KV cache, GPU and PCIe).

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

GPU Profiler for AI agents

A static performance model for LLM inference: given (model, GPU, interconnect, workload), instantly answer VRAM feasibility, latency(TTFT, TPOT, ITL), and throughput.

A tool for benchmarking LLMs on Modal