AI RESEARCH
Safety-Oriented Evaluation of Language Understanding Systems for Air Traffic Control
arXiv CS.CL
•
ArXi:2605.11769v1 Announce Type: new Air Traffic Control (ATC) is a safety-critical domain in which incorrect interpretation of instructions may lead to severe operational consequences. While large language models (LLMs) nstrate strong general performance, their reliability in operational ATC environments remains unclear. Existing evaluation approaches, largely based on aggregate metrics such as F1 or macro accuracy, treat all errors uniformly and fail to account for the asymmetric consequences of high-risk semantic mistakes (e.g., incorrect runway identifiers or movement constraints.