Understanding the Process of Data Labeling in Cybersecurity

Many domains now leverage the benefits of Machine Learning (ML), which promises solutions that can autonomously learn to solve complex tasks by training over some data. Unfortunately, in cyberthreat detection, high-quality data is hard to come by. Moreover, for some specific applications of ML, such data must be labeled by human operators. Many works "assume" that labeling is tough/challenging/costly in cyberthreat detection, thereby proposing solutions to address such a hurdle. Yet, we found no work that specifically addresses the process of labeling from the viewpoint of ML security practitioners. This is a problem: to this date, it is still mostly unknown how labeling is done in practice---thereby preventing one from pinpointing "what is needed" in the real world.