Learning Perturbations to Extrapolate Your LLM

ArXi:2605.13284v1 Announce Type: cross Recent advancements in large language models nstrate that injecting perturbations can substantially enhance extrapolation performance. However, current approaches often rely on discrete perturbations with fixed designs, which limits their flexibility. In this work, we propose a framework where token prefixes are perturbed by a learnable transformation of a continuous latent vector within an embedding space.