I‐SI: Scalable Architecture for Analyzing Latent Topical‐Level Information From Social Media Data

Abstract We present a general visual analytics architecture that is designed and implemented to effectively analyze unstructured social media data on a large scale. Pipelined on a high‐performance cluster configuration, MPI processing, and interactive visual analytics interfaces, our architecture, I‐SI , closely integrates data‐driven analytical methods and user‐centered visual analytics. It creates a coherent analysis environment for identifying event structures, geographical distributions, and key indicators of emerging events. This environment supports monitoring, analyzing, and responding to latent information extracted from social media. We have applied the I‐SI architecture to collect social media data, analyze the data on a large scale and uncover the latent social phenomena. To demonstrate the efficacy and applicability of I‐SI, we describe several social media use cases in multiple domains that were evaluated by experts. The use cases demonstrate that I‐SI can benefit a range of users by constructing meaningful event structures and identifying precursors to critical events within a rich, evolving set of topics.