gbram1/embers — GitHub trending stats & insights

Reach 125K+ monthly visitors

gbram1/embers

Serverless-GPU LLM serving: scale-to-zero with fast GPU snapshot/restore (cuda-checkpoint), multi-tenant packing, and an OpenAI-compatible API — built on vLLM.

Visit GitHub

Python

2 contributors

Apache License 2.0

Social mentions

Recent discussions about this repository across the web

It also snapshots tensor-parallel models across 2 GPUs (cold ~103s → restore ~16s) — the part NVIDIA's Dynamo Snapshot preview can't do yet. Same cuda-checkpoint mechanism underneath. Difference:…

@GabrielBram12 · x.com

Repository activities

repository's daily and monthly activities across stars, forks, merged PRs, issues, and closed issues

Data is not available yet

Recent activity data for stars, forks, merged PRs, issues, and closed issues will appear here once available