peonist-ai/halogen-flash-server

The fastest way to run Qwen3.8 27B on Strix Halo (gfx1151)

LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context

Strix Halo inference engine. Qwen Flash Next Q4_K_XL: 1,628.52pp, 59.41tg single user, 157.22 tok/s 8 users; Qwen27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2

Run Qwen3.8-Flash-Next on a single AMD Strix Halo using Halogen Flash Server

The Open Source Strix Halo AI engine. Qwen 3.8 Flash Next Q4_K_XL: 1,801.01 pp, 73.17 tg single user, 194.37 tok/s 8 users; Qwen 3.8 27B Q4_K_XL: 656.33pp, 70.56tg tok/s single user with DFlash2

GPU + XDNA2 NPU inference for AMD Strix Halo (gfx1151): Qwen3.6-35B-A3B, Qwen3.8-27B, Qwen3.8 Flash-Next, Flint and GLM-5.3 Flash (321B MoE). NPU prefill up to +28%, Flint 4.1k tok/s prefill, MTP / DFlash2 / auto speculative decoding up to 159 tok/s on structured text.

Local inference engine for Qwen3.8-Flash-Next on NVIDIA Blackwell

A custom inference engine tuned for Qwen4 architecture. Currently runs Qwen3.8-Flash-Next at 15+ tok/s on 16GB of VRAM and 30 GB of RAM

llama.cpp modified to work with Strix Halo + dGPU as an accelerator, Dual Strix Halo Machines, or both