AI RESEARCH
SlimQwen: Exploring the Pruning and Distillation in Large MoE Model Pre-training
arXiv CS.AI
•
ArXi:2605.08738v1 Announce Type: cross Structured pruning and knowledge distillation (KD) are typical techniques for compressing large language models, but it remains unclear how they should be applied at pre