siris9476/pulsarforge

SSD-streaming inference engine for giant MoE models (Rust + CUDA). GLM 5.2 743B at 2 tok/s and Hy3 295B at 7 tok/s on two consumer 16GB GPUs. Zero-config multi-GPU: measures PCIe bandwidth, places attention and hot experts where they fit.

CPU-optimized LLM inference engine (C)

Frontier-class LLM inference on a laptop CPU — gpt-oss:20b at ~110 tok/s on Apple M4 Max, 7.5x llama.cpp, no GPU. From-scratch NEON/SME kernels in Rust.

Pure Rust Inference Engine

Artifact for paper "PIM is All You Need: A CXL-Enabled GPU-Free System for LLM Inference", ASPLOS 2025

~1300 line, minimal, extensible LLM inference engine built from scratch.

Zig INferenCe Engine — Local LLM inference on AMD GPUs and Apple Silicon

LLM inference in C/C++