AI RESEARCH
FLUID: Continuous-Time Hyperconnected Sparse Transformer for Sink-Free Learning
arXiv CS.AI
•
ArXi:2605.04421v1 Announce Type: cross Continuous-time (CT) Transformers improve irregular and long-range modeling over CT-RNNs by exploiting inputs or outputs embeddings with continuous dynamics. However, the core scaled-dot-product-attention (SDPA) mechanism remains inherently discrete. We propose FLUID (Flexible Unified Information Dynamics), a CT Transformer that incorporates continuous dynamics directly into the attention computation by replacing it with Liquid Attention Network