AI RESEARCH
[D] Releasing a professional MQM-annotated MT dataset (16 lang pairs, 48 annotators)
r/MachineLearning
•
Hey all, We've been doing translation quality evaluation work and decided to open-source one of our annotated datasets. Most MT test sets out there have either crowdsourced (noisy) annotations or are locked behind paywalls - we wanted to put something out with proper professional linguist annotations. What's in it: 362 translation segments 16 language pairs 48 professional linguists (not crowdsourced) Full MQM error annotations (category, severity, span) Multiple annotators per segment for IAA analysis The methodology follows WMT guidelines - same error typology, same severity levels.