Defending active learning against adversarial inputs in automated document classification

Business and government operations generate large volumes of documents to be categorized through machine learning techniques before dissemination and storage. One prerequisite in such classification is to properly choose training documents. Active learning emerges as a technique to achieve better accuracy with fewer training documents by choosing data to learn and querying oracles for unknown labels. In practice, such oracles are usually human analysts who are likely to make mistakes or, in some cases, even intentionally introduce erroneous labels for malicious purposes. We propose a risk-factor based strategy to defend active-learning-based document classification against human mistakes or adversarial inputs. We show that the proposed strategy can substantially alleviate the damage caused by malicious labeling. Our experimental results demonstrate the effectiveness of our defense strategy in terms of maintaining accuracy against adversaries.