Neroued/ninfer

High-performance single-GPU inference for selected model checkpoints and GPUs.

High-performance single-GPU inference for selected model checkpoints and GPUs.

High-performance single-GPU inference for selected model checkpoints and GPUs.

Hyper optimised Qwen3.8-27B inference on one RTX 3090: ReplaySSM, MTP3, reasoning effort, C1-C8 batching, and native Windows and Linux builds.

NInfer port for Qwen 3.8 27B on RTX 4090

ninfer-ext: Forked for speed.

One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF block formats, ternary Bonsai 2, device route profiles, MTP and DFlash2.

Native Windows port of NInfer engine for RTX 5090. Features Qwen3.8-27B with QUASAR and NInfer models, MTP/DFlash2 with vision and 262,144 context

A custom inference engine tuned for Qwen4 architecture. Currently runs Qwen3.8-Flash-Next at 15+ tok/s on 16GB of VRAM and 30 GB of RAM