nisesen/inferenceinquire

Live truecolor terminal dashboard for a locally running LLM: tok/s, TTFT, GPUs, context, MTP acceptance, layer and real expert routing (llama.cpp)

Single-file, dependency-free live dashboard for local llama.cpp/vLLM serving — GPU, throughput, KV/ctx, model library. Stdlib Python + one local-first HTML page.

A top for local LLMs: memory, paging, GPU and tokens/s for oMLX, Ollama, llama.cpp, vLLM and more

Real-time hardware and LLM inference monitoring — GPU, CPU, memory, and vLLM metrics streamed to a dashboard.

Live dashboard: fire N parallel streaming coding-agent runs at any OpenAI-compatible endpoint — per-run TTFT/tok-s/E2E, real kill switch, per-GPU metrics, shareable summary. Zero deps.

sparkDash ⚡ — Multi-DGX Spark Monitoring Dashboard

The htop for LLM inference see exactly where every GB of VRAM goes and get measured quantization savings.

A physics-grounded, cost-aware optimization loop for vLLM

Local LLM Inference Speed Test Tool