arsenyivanov/e-moe

[EMNLP 2026 (Oral)] Multi-Block Diffusion Language Models

Implementation of LatentMoE,Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts (Elango et al., NVIDIA 2026) — in Pytorch. A single-file, dependency-light layer you can drop in place of a standard MoE FFN.

The first continuous diffusion language model that rivals discrete counterparts on standard language modeling benchmarks like LM1B and OpenWebText.

Official implementation of "Hierarchical Continuous Diffusion Language Models"

Pruning and Distilling Mixture-of-Experts into Dense Language Models

RND1: Scaling Diffusion Language Models

Instella MoE is a fully open state-of-the-art Mixture-of-Experts language model, trained end-to-end on AMD Instinct™ GPUs.

[NeurIPS 2026] DMax: Aggressive Parallel Decoding for dLLMs