AI RESEARCH
Beyond In-Distribution Success: Scaling Curves of CoT Granularity for Language Model Generalization
arXiv CS.CL
•
ArXi:2502.18273v2 Announce Type: replace Generalization to novel compound tasks under distribution shift is important for deploying transformer-based language models (LMs). This work investigates Chain-of-Thought (CoT) reasoning as a means to enhance OOD generalization. Through controlled experiments across several compound tasks, we reveal three key insights: (1) While QA-trained models achieve near-perfect in-distribution accuracy, their OOD performance degrades catastrophically, even with 10000k.