Drawback of Enforcing Equivariance and its Compensation via the Lens of Expressive Power

ArXi:2512.09673v3 Announce Type: replace-cross Equivariant neural networks encode the intrinsic symmetry of data as an inductive bias, which has achieved impressive performance in wide domains. However, the understanding to their expressive power remains premature. Focusing on 2-layer ReLU networks, this paper investigates the impact of enforcing equivariance constraints on the expressive power. By examining the boundary hyperplanes and the channel vectors, we constructively nstrate that enforcing equivariance constraints could undermine the expressive power.