Mixing Synthetic Data for Small-Data Document Image Classification

This study investigates training and data generation strategies for document error classification in low-data, imbalanced scenarios. We evaluate four models: SVM with SIFT features, a standard CNN, a contrastively pretrained CNN, and a ResNet50 with transfer learning. These are compared across sampled, upsampled, synthetic, and mixed datasets using accuracy and Cohen’s Kappa. Contrary to expectations, both contrastive pretraining and transfer learning offered limited benefits, highlighting the challenges of applying large-scale, non-document-specific pretraining to document errors. A supervised CNN trained on mixed data, combining synthetic and upsampled images, achieved the best results, improving accuracy by up to 7% and Kappa score by up to 30%. In contrast, using synthetic data alone resulted in a performance decrease, while increasing dataset size had minimal impact. These findings emphasize that domain-specific data balancing outperforms pretrained models in low-resource document classification. Additionally, mixing synthetic with upsampled data offers a cost-effective and hardware-efficient approach to address data scarcity and class imbalances.