PromptAug: Fine-grained Conflict Classification Using Data Augmentation

Given the rise of interpersonal conflict behaviours on social media, classification models are essential for detecting harmful behaviours. Following the garbage-in-garbage-out maxim, machine learning performance depends on training data quality. However, quality labelled data, especially for nuanced tasks like identifying conflict behaviours, is limited, expensive, and difficult to obtain. Additionally, as social media platforms restrict access to research data, text data augmentation is gaining attention as an alternative means of generating training data. Augmenting conflict-related data poses challenges due to Large Language Model (LLM) guardrails restricting generation of offensive content. This paper introduces PromptAug, an LLM-based data augmentation method. PromptAug achieves significant improvements of 2% in both accuracy and F1-score in conflict and emotion classification tasks. To evaluate PromptAug against other data augmentation methods, we conduct a robust set of experiments, including: extreme data scarcity scenarios, quantitative diversity analysis, and a qualitative thematic analysis. The thematic analysis identifies four error patterns in augmented text: Linguistic Fluidity, Humour Ambiguity, Augmented Content Ambiguity, and Augmented Content Misinterpretation. Overall, this work presents PromptAug as an effective method for augmenting training data in sensitive tasks like fine-grained conflict detection, offering a robust, interdisciplinary evaluation grounded in both natural language processing and social science methodology.