AI RESEARCH
Discrete Tokenization Unlocks Transformers for Calibrated Tabular Forecasting
arXiv CS.LG
•
ArXi:2603.07448v1 Announce Type: new Gradient boosting still dominates Transformers on tabular benchmarks. Our tokenizer uses a deliberately simplistic discretized vocabulary so we can highlight how even basic tokenization unlocks the power of attention on tabular features, yet it already outperforms tuned gradient boosting when combined with Gaussian smoothing. Our solution discretizes environmental context while smoothing labels with adaptive Gaussians, yielding calibrated PDFs. On 600K entities (5M.