An Ensemble Method for Variable-Grain Political Ideology Classification on Social Media Texts

During the early 21st century, the rise of social media has significantly affected all areas of social existence, including politics. Among other phenomena, this resulted in the development of combative online discourses such as toxicity and political polarization, observable on many social media platforms, such as X (formerly known as Twitter). While this novel pragmatics of online discourse has been widely noted and studied from both the computational and the discourse analysis standpoints, the ideological contents of the discourse have been less scrutinized. In this paper we present an ensemble deep learning classification pipeline able to categorize the political or non-political nature of a short text, as well as the left-right ideological orientation of the text on two separate granularity scales–3-point and 7-point. These models were trained on an existing political ideology dataset which we have compiled from publicly available news websites exhibiting a certain political affiliation. Using BERT-based classifiers, an F1 score of 96.39% is obtained for the political classification subtask, whereas for the coarse-grained and fine-grained ideology classification subtasks, F1 scores of 92.84% and 90.33% are obtained, respectively, on the validation dataset with 5-fold cross-validation. The model was then used to predict the political or non-political nature and the ideological orientation of a sample of 2.6M tweets retrieved from the #Election2020 dataset, which was followed up with an n-gram analysis of the classification results, using s-BERT embeddings, PCA dimensionality reduction and DBSCAN clustering to reveal discussion topics relevant to different ideological orientations. We analyze the results and show the prevalence of each ideology in the data and use SHAP explainers to verify the accuracy of the predictions, revealing stylistic features characteristic of each political orientation that can prove helpful in distinguishing politically-charged conversations, especially extremist discourses.