llama.cpp optimized for NVIDIA Tesla V100 (SM70). Qwen3.8-Flash-Next (MoE) with MTP on 1/2/3/6x V100-16GB at 262K context (6 GPUs: prefill 3402 tok/s at 200K; 2 GPUs on one socket: 2152 tok/s). Qwen3.8-27B with DFlash2 on 1-6 GPUs, up to 512K context. Multimodal, thinking, concurrent serving, built-in web UI.
llama.cpp tuned for 2x Tesla P100 (sm_60). Decode 1.86x, 3-4x with MTP (up to ~70 t/s on code). Prefill 2x, full 262k context with vision. More numerically accurate than upstream.
One line of NInfer for the Blackwell, Ada Lovelace, Ampere, Turing and Volta NVIDIA GPUs, consolidated from the forks that carry it: GGUF block formats, ternary Bonsai 2, device route profiles, MTP and DFlash2.