AI RESEARCH

Three limitations I keep hitting with retrieval-augmented generation in production and I'm running out of ideas [D]

r/MachineLearning

I've had a RAG system running in production for a few months now (legal domain, German regulatory documents). It handles 80% of queries well but there are three patterns where it fails predictably and I haven't found clean solutions. The scatter problem. Some questions need information from 8-10 different documents where each one contributes just a small piece. Vector search finds chunks related to the query but not chunks related to each other.