Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective

ArXi:2604.23267v1 Announce Type: cross Large language models (LLMs) operate in two fundamental learning modes - fine-tuning (FT) and in-context learning (ICL) - raising key questions about which mode yields greater language proficiency and whether they differ in their inductive biases. Prior studies comparing FT and ICL have yielded mixed and inconclusive results due to inconsistent experimental setups.