AI RESEARCH
IntSeqBERT: Learning Arithmetic Structure in OEIS via Modulo-Spectrum Embeddings
arXiv CS.LG
•
ArXi:2603.05556v1 Announce Type: new Integer sequences in the OEIS span values from single-digit constants to astronomical factorials and exponentials, making prediction challenging for standard tokenised models that cannot handle out-of-vocabulary values or exploit periodic arithmetic structure. We present IntSeqBERT, a dual-stream Transformer encoder for masked integer-sequence modelling on OEIS. Each sequence element is encoded along two complementary axes: a continuous log-scale magnitude embedding and sin/cos modulo embeddings for 100 residues (moduli $2$--$101$), fused via FiLM.