AI RESEARCH
MiCA Learns More Knowledge Than LoRA and Full Fine-Tuning
arXiv CS.LG
•
ArXi:2604.01694v1 Announce Type: new Minor Component Adaptation (MiCA) is a novel parameter-efficient fine-tuning method for large language models that focuses on adapting underutilized subspaces of model representations. Unlike conventional methods such as Low-Rank Adaptation (LoRA), which target dominant subspaces, MiCA leverages Singular Value Decomposition to identify subspaces related to minor singular vectors associated with the least significant singular values and constrains the update of parameters during fine-tuning to those directions.