CARROT: A Learned Cost-Constrained Retrieval Optimization System for RAG

ArXi:2411.00744v2 Announce Type: replace-cross Large Language Models (LLMs) have nstrated impressive ability in generation and reasoning tasks but struggle with handling up-to-date knowledge, leading to inaccuracies or hallucinations. Retrieval-Augmented Generation (RAG) mitigates this by retrieving and incorporating external knowledge into input prompts. In particular, due to LLMs' context window limitations and long-context hallucinations, only the most relevant "chunks" are retrieved.