FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff

ArXi:2602.08040v3 Announce Type: replace-cross Deep neural networks trained on nonstationary data must balance stability (i.e., retaining prior knowledge) and plasticity (i.e., adapting to new tasks). Standard reinitialization methods, which reinitialize weights toward their original values, are widely used but difficult to tune: conservative reinitializations fail to re plasticity, while aggressive ones erase useful knowledge. We propose FIRE, a principled reinitialization method that explicitly balances the stability-plasticity tradeoff.