AI RESEARCH
COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation
arXiv CS.LG
•
ArXi:2507.07580v3 Announce Type: replace Recent studies suggest that context-aware low-rank approximation is a useful tool for compression and fine-tuning of modern large-scale neural networks. In this type of approximation, a norm is weighted by a matrix of input activations, significantly improving metrics over the unweighted case. Nevertheless, existing methods for neural networks suffer from numerical instabilities due to their reliance on classical formulas involving explicit Gram matrix computation and their subsequent inversion.