AI RESEARCH
QFT: Quantized Full-parameter Tuning of LLMs with Affordable Resources
arXiv CS.LG
•
ArXi:2310.07147v3 Announce Type: replace-cross Large Language Models (LLMs) have showcased remarkable impacts across a wide spectrum of natural language processing tasks. Fine-tuning these pretrained models on downstream datasets provides further significant performance gains; however, this process typically requires a large number of expensive, high-end GPUs. Although there have been efforts focused on parameter-efficient fine-tuning, they cannot fully unlock the powerful potential of full-parameter fine-tuning.