kacper-daftcode/vLLM-Moet

LvLLM is a special NUMA extension of vllm that makes full use of CPU and memory resources, reduces GPU memory requirements, and features an efficient GPU parallel and NUMA parallel architecture, supporting hybrid inference for MOE large models.

PyTorch library for cost-effective, fast and easy serving of MoE models.

Mixture-of-experts (MoE) training megakernel for NVL72s

~1300 line, minimal, extensible LLM inference engine built from scratch.

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4

Run MoE models beyond VRAM on AMD, NVIDIA & Intel GPUs. Vulkan-powered, portable Windows app with automatic setup. | 让游戏显卡跑起超显存大模型，支持 AMD、NVIDIA、Intel，免安装、自动配置。只需13MB，即下即用！

A Flexible Framework for Experiencing Heterogeneous LLM Inference/Fine-tune Optimizations

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported