sybil-solutions/qwen-serve

Deployment kit: Qwen3.8-27B EXL3 3.5bpw target + DFlash2 EXL3 5.0bpw speculative draft — launcher, config, model cards

Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux builds.

Qwen3.8 27B on SGLang for DGX Spark

Qwen3.8-27B on 16-32 GB Nvidia GPUs one-click install for Windows / Linux

Qwen3.8 Flash at 74 tok/s on a single DGX Spark / GB10. vLLM + MTP recipe, 212 tok/s at 8 streams.

Serve large Qwen models fast on the GPUs you actually own. Qwen3.8-27B on a single 24 GB card with vLLM: 127 tok/s single-user (381 when the answer quotes the prompt), ~1,035 tok/s at 64 concurrent, 150k-262k context. vLLM patches, requant pipeline, benchmarks.

qwen38-flash-next recipe for solo GB10

Qwen3.8 Flash Next on one DGX Spark (TensorFold)

NInfer port for Qwen 3.8 27B on RTX 4090