DeepSeek Updated their repo DeepGEMM testing Mega MoE
r/LocalLLaMA
•
Open Source AI
AI Research
Mega MoE is still under development and optimizations, stay tuned and optimization ideas are welcome! Disclaimer: this release is only related to DeepGEMM's development, has nothing to do with internal model release. P4 + Mega MoE + Distributed Communication + Blackwell Adaptation + HyperConnection