Tightening convex relaxations of trained neural networks: a unified approach for convex and S-shaped activations

ArXi:2410.23362v2 Announce Type: replace-cross The non-convex nature of trained neural networks has created significant obstacles in their incorporation into optimization models. In this context, Anderson provided a framework to obtain the convex hull of the graph of a piecewise linear convex activation function composed with an affine function; this effectively convexifies activations such as the ReLU together with the affine transformation that precedes it.