An Analysis of Active Learning Algorithms using Real-World Crowd-sourced Text Annotations

ArXi:2604.23290v1 Announce Type: new Active learning algorithms automatically identify the most informative samples from large amounts of unlabeled data and tremendously reduce human annotation effort in inducing a machine learning model. In a conventional active learning setup, the labeling oracles are assumed to be infallible, that is, they always provide correct answers (in terms of class labels) to the queried unlabeled instances, which cannot be guaranteed in real-world applications.