Evaluation of Elasticsearch Ecosystem Including Machine Learning Capabilities

Earlier methods of signature based threat detection are no longer enough to detect threats.Fencing your network and access is also ineffective in stopping malicious actors because the IT infrastructure is continuously being shifted to the cloud.Continuous data collection, monitoring and watching out for any malicious behaviors can detect zero day or unknown threats as well.This paper focuses on one of the most important and widely used collections of such tools which are built around Elasticsearch (ES).This paper explains Elasticsearch and its ecosystem of tools like Filebeat and Kibana.A test bed is set up consisting of Apache Web server, Elasticsearch, Filebeat and Kibana.Also the machine learning (ML) capabilities of Elasticsearch are demonstrated with manually injected anomalies in the metric data collected for the web server.