sybil-solutions/glm-flash-lite

GLM-5.3-Flash EXL3 on 2x DGX Spark with TensorFold

Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux builds.

High-performance single-GPU inference for selected model checkpoints and GPUs.

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

Run the full 2.78-trillion-parameter Kimi K3 model, DeepSeek V4.1 Flash or GLM-5.3-Flash beyond available RAM by streaming activated weights directly from NVMe. A dependency-free, embeddable C inference engine.

Community recipes for serving LLMs on RTX 3090/4090/5090 CUDA gpus. Multi-engine (vLLM, sglang, llama.cpp and other custom engines) and model-agnostic. Currently shipping Qwen3.8-27B Qwen3.6 35B Gemma 4 26B Gemma 4 31B configs for 1×/2×/4x/8x cards.

GLM-5.3 Flash EXL3 for 2-4x DGX Sparks

The fastest way to run Qwen3.8-Flash-Next on Strix Halo (gfx1151)

DeepSeek v4.1 Flash EXL3 2.9 bpw for 2x DGX Sparks