nsbradley88/talus-public

LLM inference in C/C++

RDNA-native LLM inference engine in Rust.

LLM inference engine written from scratch in HIP for AMD Strix Halo (gfx1151) and Qwen3.8-Flash-Next - ~50 tokens/s out to 492k context

llama.cpp modified to work with Strix Halo + dGPU as an accelerator, Dual Strix Halo Machines, or both

Performance-tuned llama.cpp for AMD Strix Halo (gfx1151): FA + MoE-prefill fixes with a bundled current Mesa driver. Vulkan and HIP; portable dir, Docker, and distrobox.

Local inference that behaves like a frontier API: Qwen3.8-Flash-Next on AMD Strix Halo plus small NPU models, over the OpenAI and Anthropic APIs on one port.

A deterministic inference engine for DeepSeek-V4.1-Flash 1017 and Qwen3.8-Flash-Next on a single AMD Strix Halo 128 GB

A high-throughput and memory-efficient inference and serving engine for LLMs - Optimized for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60

Pure Rust Inference Engine