AI RESEARCH
Hardware-Efficient Softmax and Layer Normalization with Guaranteed Normalization for Edge Devices
arXiv CS.LG
•
ArXi:2604.23647v1 Announce Type: cross In Transformer models, non-GEMM (non-General Matrix Multiplication) operations -- especially Softmax and Layer Normalization (LayerNorm) -- often dominate hardware cost due to their nonlinear nature. To address this, previous approximation studies mainly target rank-oriented tasks, which is acceptable for classification. However, edge Natural Language Processing (NLP) applications and edge generative AI are largely evaluated based on score-oriented tasks, so normalization-guaranteed non-GEMM operations are essential.