kyegomez/Latent-MoE

Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4

Dist-MoE implements Speed-of-Light distributed Mixture-of-Experts forward and backward kernels. They are highly optimized and fused comm-compute kernels for training and inference on SM100+ GPUs

Tiny-MoE is a lightweight Mixture-of-Experts language model built entirely from scratch in native PyTorch and trained end-to-end on Kaggle using free 2× NVIDIA T4 GPUs. The project implements modern LLM techniques—including MLA, RoPE, YaRN, streaming pre-training, and efficient inference—without relying on existing model implementations.

PyTorch library for cost-effective, fast and easy serving of MoE models.

Accelerating MoE with IO and Tile-aware Optimizations

Pytorch implementation of "E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models" (NeurIPS 2026 DiffuLM Workshop).

The official implementation of the paper "Towards Efficient Mixture of Experts: A Holistic Study of Compression Techniques (TMLR)".

Pruning and Distilling Mixture-of-Experts into Dense Language Models