Moonshot AI just open-sourced MoonEP, a system designed to keep Mixture-of-Experts workloads balanced across GPUs. The useful idea is larger than GPUs. MoonEP keeps the model's logical expert choice,…

one might want to yap like pp is too ugly, tp is scam, … but you may need pp at this scale. huge efforts for infra!

china just published the closest thing yet to an open frontier AI factory. Kimi K3 has 2.8 trillion total parameters, but activates 104 billion for each token. Think of 896 specialist brains inside…