EfficientMoE/MoE-Infinity

Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4

Accelerating MoE with IO and Tile-aware Optimizations

MoonEP: A Perfectly Balanced Expert Parallelism Library via Dynamic Redundant Experts

Production-ready MoE load balancing via real-time expert replication

Run MoE models bigger than your RAM. Frontier-size MoE on a 12 GB phone, lossless, on stock llama.cpp

A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official (NV)FP4 checkpoint's quality on consumer Blackwell cards

Compact and Agent-Native MoE Training System

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

FlashInfer: Kernel Library for LLM Serving