So, vLLM takes approximately ten trillion years to start. On the other hand... 208 tok/s from Qwen3.8-27B may be worth it.

一张24GB的RTX 3090，真能把27B模型服务做出可用指标？ 它用int8张量核、fp16状态和推测解码，做到64并发约1035 tok/s、单路约114至131 tok/s，最长约26万上下文。量化脚本、补丁、校验、Docker和基准都齐。代价也明确：超长上下文KV有损，部分模式牺牲并发，补丁绑定vLLM版本，升级维护不能按零成本算。

The quant alone is not why its fast. The syv-ai recipe requants lm_head, embeddings and the MTP module to int8, in place, cpu only. That frees 2.6 GB per card: KV pool grows from 282K to 362K tokens…