Verified record · from the answer index

moe-training

Train Mixture of Experts (MoE) models using DeepSpeed or HuggingFace. Use when training large-scale models with limited compute (5× cost reduction vs dense models), implementing sparse architectures like Mixtral 8x7B or DeepSeek-V3, or scal

skill:orchestra-research/ai-research-skills/moe-training · published by orchestra-research · source repository