Universal Smoothness via Bernstein Polynomials: A Constructive Approximation Approach for Activation Functions

ArXi:2605.02591v1 Announce Type: new The efficacy of deep neural networks is heavily reliant on the design of non-linear activation functions, yet existing approaches often struggle to balance optimization stability with computational efficiency. While piecewise linear functions offer inference speed, they suffer from optimization instability due to non-differentiability at the origin, whereas smooth counterparts typically incur significant computational overhead through their reliance on transcendental operations.