r0b0tlab/qwen38-exl3-dflash2

Deployment kit: Qwen3.8-27B EXL3 3.5bpw target + DFlash2 EXL3 5.0bpw speculative draft — launcher, config, model cards

Easy local setup for Qwen3.8-27B Abliterated: 256K context and measured 240-300 tokens/s generation on an RTX 5090.

A llama.cpp fork with TurboQuant/TCQ KV cache, fused Qwen MTP speculative decoding, and multi-GPU long-context serving fixes.

Qwen3.8-Flash-Next on 2× RTX 3090: with 128 GB RAM up to 4,191 tok/s prefill · 111.5 tok/s decode (131K prompt), full 256K window at 2,865 tok/s; with 64 GB RAM 3,410 tok/s prefill · 84 tok/s decode. New: opt-in uncensored mode (runtime abliteration, no new weights).

NInfer port for Qwen 3.8 27B on RTX 4090

Optimized NINFER runtime and reproducible Qwen3.8-27B true-128K inference for NVIDIA RTX 5080 16GB

Qwen3.8-27B-GSQ3 on one RTX 4080: 100k context with rk4v4-e8, up to 2720 tok/s prefill and 262 tok/s generation

Qwen3.8-27B on one RTX 4090: full native 262K context via E8 4-bit KV, up to 149 tok/s code decode with MTP3, sm_89-retuned attention prefill, vision, llama.cpp-compatible /metrics + /slots