Which LLMs actually fail when domain knowledge is buried in long documents?
r/LocalLLaMA
•
Generative AI
Open Source AI
I’ve been testing whether frontier LLMs can retrieve expert industrial knowledge (sensor-failure relationships from ISO standards) when the relevant information is buried inside long documents. The interesting pattern so far: DeepSeek V3.2 answers the questions correctly in isolation but fails when the same question is embedded in a long context. Gemma 3 27B fails on the domain knowledge itself, regardless of context.