AI RESEARCH
Contrast-Enhanced Gating in GRUs for Robust Low-Data Sequence Learning
arXiv CS.LG
•
ArXi:2402.09034v3 Announce Type: replace Activation functions govern how recurrent networks regulate and transmit information across temporal dependencies. Despite advances in sequence modelling, gated recurrent units (GRUs) still depend on the standard sigmoid and tanh nonlinearities, which can produce weak gate separation and unstable learning, particularly when