CASE-Lab-UMD/Unified-MoE-Compression

Pruning and Distilling Mixture-of-Experts into Dense Language Models

Implementation of LatentMoE,Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts (Elango et al., NVIDIA 2026) — in Pytorch. A single-file, dependency-light layer you can drop in place of a standard MoE FFN.

Post-Trained MoE Can Skip Half Experts via Self-Distillation

Tutel MoE: Optimized Mixture-of-Experts Library, Support GLM/Kimi/Gpt/DeepSeek/Qwen using FP8/NVFP4/MXFP4

Expert Specialized Fine-Tuning

For releasing code related to compression methods for transformers, accompanying our publications

Official implementation of LittleBit (NeurIPS 2025) and its follow-up LittleBit-2 (ICML 2026)

RCO: Optimization of Non-Decomposable Objectives under Exact Budget Constraints for LLM Quantization and Pruning

[TKDE'25] The official GitHub page for the survey paper "A Survey on Mixture of Experts in Large Language Models".