AI RESEARCH
IGLU: The Integrated Gaussian Linear Unit Activation Function
arXiv CS.LG
•
ArXi:2603.06861v1 Announce Type: new Activation functions are fundamental to deep neural networks, governing gradient flow, optimization stability, and representational capacity. Within historic deep architectures, while ReLU has been the dominant choice for the activation function, modern transformer-based models increasingly are adopting smoother alternatives such as GELU and other self-gated alternatives. Despite their empirical success, the mathematical relationships among these functions and the principles underlying their effectiveness remains only partially understood. We.