What question did this study set out to answer?

This research aims to analyze stylistic convergence in Japanese technical writing using various language models.

March 25, 2026Open Access

AI Text Slop: A Quantitative Study of Stylistic Convergence Across Six Language Models in Japanese Technical Writing

Key Points

This research aims to analyze stylistic convergence in Japanese technical writing using various language models.
Conducted a quantitative analysis of 180 samples from six language models.
Measured samples using 16 linguistic pattern indicators.
Compared scores of RLHF-aligned models to open-source alternatives.
Commercial models showed significantly higher scores on stylistic indicators (Cohen's d = 1.01).
Vocabulary-level and structural indicators revealed a dissociation known as the 'Swallow Paradox'.
Human-written articles outperformed AI models in structural metrics, indicating cultural influences.

Abstract

We present the first quantitative analysis of stylistic convergence in Japanese technical blog writing across six LLMs (Claude Sonnet 4, GPT-4o, Qwen 3.5-4B/9B, Swallow-20B, Llama 3.2-1B) using 180 samples measured with 16 linguistic pattern indicators. Key findings: (1) RLHF-aligned commercial models score significantly higher than open-source alternatives (Cohen's d = 1.01); (2) vocabulary-level and structural indicators dissociate (the "Swallow Paradox"); (3) human Qiita articles score higher than AI on structural metrics, revealing cultural confounding. All data, code, and analysis scripts are included.

AI Text Slop: A Quantitative Study of Stylistic Convergence Across Six Language Models in Japanese Technical Writing

Key Points

Abstract

Cite This Study