AI RESEARCH
Two-Stage Optimizer-Aware Online Data Selection for Large Language Models
arXiv CS.AI
•
ArXi:2604.00001v1 Announce Type: cross Gradient-based data selection offers a principled framework for estimating sample utility in large language model (LLM) fine-tuning, but existing methods are mostly designed for offline settings. They are therefore less suited to online fine-tuning, where data arrives sequentially, sample utility is step-dependent, and the effective update geometry is shaped by adaptive optimizers. We propose an optimizer-aware framework for gradient-based online data selection and reweighting in LLM fine-tuning.