Easy, fast, and cheap LLM serving for everyone 📦 vllm ⭐ 80,872 🐍 Python #LLM #PyTorch #Python 🔗

Most people trying to build with AI hit the same wall: «API costs.» Experiment a little. Test a few agents. Run some workflows. Suddenly your credits are gone. That’s why repos like this spread fast.…

Are there any vLLM code maintainers active on X? I have two small PRs that have not received any feedback yet. I’d really appreciate any review, comments, or guidance on what might be blocking them.

Flash-MaxSim 🚀 FlashAttention killed the materialized attention matrix in 2022. The same Lq×Ld matrix still lives in every ColBERT/ColPali pipeline — 21 GB for ColPali @ 10K docs, built only to be…

午后开源工具链速递 ☕️ 🔗 vLLM v0.21.0（5/15）：KV Offload + Hybrid Memory Allocator 上线，推测解码支持 thinking budget，Blackwell GPU 上 DeepSeek-R1 走 TOKENSPEED_MLA 后端 ⚡ Unsloth v0.1.405-beta（5/18）：MTP 推测解码让 GGUF 推理 2x…

让Claude Opus 4.7 Max点评了一下 DeepSeek V4 Flash 的生成质量，评测的产物在 用我的 vLLM fork 跑的 覆盖度齐全：nomtp / mtp2 / mtp1 各 315 个 .md（17 case × 3 mode × 3 round × 2 lang）。开读代表性样本—— 读完代表性样本，给你认真说下我的主观评价。 产物覆盖度 ✓ 315…

🐍 vllm ⭐ 80,203 stars **"vLLM : Le serveur LLM ultra-rapide, économe et accessible à tous !"** 🚀 #GitHub

Hi @vllm_project, could someone review vllm-project/vllm#41967? It impacts Gemma4 streaming tool calls with MTP/speculative decoding. Fixes are already proposed in #42006 and #42300. Maintainer…

🐾 vLLM: Fast and Easy LLM Serving for Everyone Unlock super-fast language model responses with minimal effort! Dive into state-of-the-art serving and see your models perform like never before. Learn…

Benchmark LLMs beyond just accuracy scores. Focus on operational metrics too. Consider `tokens/second`: - Measures output generation speed. - Crucial for interactive user experiences. - Impacts…