BEFT: Bias-Efficient Fine-Tuning of Language Models in Low-Data Regimes

ArXi:2509.15974v2 Announce Type: replace-cross Fine-tuning the bias terms of large language models (LLMs) has the potential to achieve unprecedented parameter efficiency while maintaining competitive performance, particularly in low-data regimes. However, the link between fine-tuning different bias terms (i.e., $\boldsymbol{b}_q$, $\boldsymbol{b}_k$, and $\boldsymbol{b}_$ in the query, key, or value projections) and downstream performance remains largely unclear to date.