What type of study is this?

This is a Quantitative Study study.

October 5, 2025Open Access

The Data-Quality Illusion: Rethinking Classifier-Based Quality Filtering for LLM Pretraining

View Full Paper

Ask AI

Bookmark

Authors

Analysis shows that classifier-based quality filtering improves downstream tasks, suggesting limits in perceived data quality.

Classifier-based quality filtering improves downstream task performance, yet may not enhance language modeling effectively.
Despite filtering based on quality, models trained show different performance trends compared to those with synthetic data.
CQF inadvertently filters out parts of the high-quality dataset, leading to paradoxical outcomes in training.
Findings challenge the belief that CQF accurately reflects a meaningful notion of data quality in pretraining.

Saada et al. (2025) studied this question.

View Full Paper

Ask AI

Bookmark