I built a system where senior lawyers can correct the AI's knowledge by leaving comments on documents. here's why it matters more than better embeddings

r/artificial
Generative AI AI Research

When I built an AI research assistant for a law firm, the feature I thought would be a nice-to-have turned out to be the one they use most. The system has an annotation feature. Any user can select text in a document and leave a comment. Something like "this interpretation was overruled by ruling X in 2024" or "this applies only to NRW, not nationally" or "our firm's position differs, see internal memo Y." Technically here's what happens. Comments are d in PostgreSQL linked to the document ID, page number, and selected text. When a query comes in, the system does two things.