Anomaly Detection on Big Data in Financial Markets

In the modern financial market, market participants use big data analytics to gain valuable insight on historical market data for better decision making. Complying with the three vs (i.e., velocity, volume and variety) of big data, the financial market is considered as a complex system comprised of many interacting high-frequency traders those make decisions based on the relative strengths of these interactions. Researchers have put substantial scholarly input to deal with these anomalies. From the big data perspective, anomaly detection in financial data has widely been ignored despite many organisations store, process and disseminate financial market data for interested customers to assist them to make informed decision abd create competitive advantages. Considering the presence of anomalies in voluminous data from myriad data sources may generate catastrophic decision through misunderstandings of market behaviour. Therefore, in this study, we applied a standard set of anomaly detection techniques, used in big data based on nearest-neighbours, clustering and statistical approaches, to detect rare anomalies present within the historical daily trading information for five years (i.e., 2009--2013) for each stock listed on the Australian Security Exchange (ASX). We also measured the performance of these anomaly detection techniques using a number of metrics to highlight the best performing algorithm. The experimental results suggest that the LOF(Local Outlier Factor) and CMGOS(Clustering-based Multivariate Gaussian Outlier Score) are the best performing anomaly detection techniques.