AI RESEARCH
Efficient and Minimax Optimal In-context Nonparametric Regression with Transformers
arXiv CS.LG
•
ArXi:2601.15014v2 Announce Type: replace-cross We study in-context learning for nonparametric regression with $\alpha$-H\"older smooth regression functions, for some $\alpha>0$. We prove that, with $n$ in-context examples and $d$-dimensional regression covariates, a pretrained transformer with $\Theta(\log n)$ parameters and $\Omega\bigl(n^{2\alpha/(2\alpha+d)}\log^3 n\bigr)$ pre