yalun753/moe-l2

LLM inference in C/C++

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

Run multiple resource-heavy Large Models (LM) on the same machine with limited amount of VRAM/other resources by exposing them on different ports and loading/unloading them on demand

Run MoE models beyond VRAM on AMD, NVIDIA & Intel GPUs. Vulkan-powered, portable Windows app with automatic setup. | 让游戏显卡跑起超显存大模型，支持 AMD、NVIDIA、Intel，免安装、自动配置。只需13MB，即下即用！

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

AirLLM 2.8T LLM inference with single 4GB GPU

PyTorch library for cost-effective, fast and easy serving of MoE models.

Runs 405B LLMs on 8GB VRAM

A vLLM patch + hand‑written SM120 SASS kernels: 2‑bit MoE experts + an FP4 "delta" cache that recovers precision — matching the official (NV)FP4 checkpoint's quality on consumer Blackwell cards