AI²: Training a Big Data Machine to Defend

We present AI2, an analyst-in-the-loop security system where Analyst Intuition (AI) is put together with state-of-the-art machine learning to build a complete end-to-end Artificially Intelligent solution (AI). The system presents four key features: a big data behavioral analytics platform, an outlier detection system, a mechanism to obtain feedback from security analysts, and a supervised learning module. We validate our system with a real-world data set consisting of 3.6 billion log lines and 70.2 million entities. The results show that the system is capable of learning to defend against unseen attacks. With respect to unsupervised outlier analysis, our system improves the detection rate in 2.92× and reduces false positives by more than 5×.