Generating realistic synthetic tabular data with integrated LLM and diffusion models

Generating realistic synthetic tabular data is a crucial task for privacy-preserving data sharing, data augmentation, and learning from limited samples. However, existing methods often struggle with small sample sizes, heterogeneous feature types, and generalization in real-world scenarios. We propose TabularMDLM, a novel diffusion-based generative framework that integrates masked language modeling to synthesize high-quality tabular data. Unlike prior work, TabularMDLM applies noise only to feature values—preserving the semantic relationship between column names and values—and leverages pre-trained language models to iteratively reconstruct masked tokens during reverse diffusion. This design enhances generation quality while ensuring structural consistency. To evaluate the framework, we conduct extensive experiments on six tabular datasets of varying sizes, domains, and feature types. We compare TabularMDLM against recent baselines, including TabDDPM, CTGAN, and CTGAN+, under a privacy-conscious setting where only synthetic data is used for training and real data for testing. We also assess performance under class imbalance to validate generalization. Results show that TabularMDLM achieves consistently strong classification performance across accuracy, precision, recall, and F1 score, outperforming baselines in both balanced and imbalanced settings. In contrast to existing methods, TabularMDLM scales to diverse data types and low-resource regimes, offering practical advantages in privacy-sensitive applications. • Introduces TabularMDLM, a generative framework combining LLMs and diffusion-inspired refinement for synthetic tabular data. • Selective masking of feature values, while preserving feature names, ensures schema integrity and richer feature interactions. • Generates more diverse and representative samples than conventional oversampling methods. • Demonstrates improved predictive accuracy and robustness, even under challenging feature compositions and severe class imbalances. • Facilitates privacy-compliant data augmentation and fairer decision-making in data-sensitive and resource-constrained scenarios.