AI RESEARCH
Temporal Task Diversity: Inductive Biases Under Non-Stationarity in Synthetic Sequence Modelling
arXiv CS.LG
•
ArXi:2605.18281v1 Announce Type: new Modern deep learning science often assumes that neural networks. How does such non-stationarity impact the inductive biases of deep learning towards models with different structural, generalisation, and safety properties? A fruitful testbed for studying inductive bias is in-context linear regression sequence modelling, where small transformers display strikingly different generalisation patterns depending on the diversity of the (fixed)