What type of study is this?

September 5, 2025Open Access

Automating Low-Fidelity Synthetic Data Generation from Metadata: A Reproducible Approach Using R

Key Points

The framework successfully generates low-fidelity synthetic data that closely mimics real datasets while ensuring privacy.
Synthetic data maintains the cardinality of categories within ±5% compared to real data, reducing identity exposure risk.
An automated pipeline utilizing metadata ensures structured synthetic data generation, streamlining researcher access.
Future refinements may enhance fidelity options and expand usability with the development of an R package.

Abstract

ObjectivesAccess to pseudonymised administrative data for research is very important for privacy preservation and regulatory compliance. Synthetic data (SD) offers a solution to address concerns related to accessibility, security and privacy. We therefore developed a scalable framework that automates the generation of low-fidelity synthetic data (LFSD) from publicly available metadata. MethodsOur R-based automated and reproducible pipeline takes metadata as input and generates SD that does not maintain the relationship between the observed variables inSM1 the original dataset. The pipeline imports metadata from datasets (data dictionaries and lookup tables), generates pre-defined pattern-based categorical, numeric and date-time variables and applies missingness on pre-defined specifications. It also randomises the SD generated, preventing disclosure; an associated quality report performs validation checks to ensure thatSM2 the structure of SD is as similar as possible to the real data (RD) whilst minimising disclosure risk. The tool has been used in successfully generating four education synthetic datasets. SM3 ResultsThe framework successfully generated LFSD that closely mimicked RD while protecting individuals’ sensitive information. To avoid any real or apparent data breaches, SD will be checked based on four parameters: labelling (to ensure SD was clearly identified as synthetic and not RD), disclosure (disclosure risk evaluation (DRE) carried out and any risks identified were mitigated), structure (ensured structure of SD is as similar as possible to RD) and documentation (differences in the structure of SD compared to RD data were documented). We found that SD maintained the cardinality of categories (within ±5%) with RD. There was no explicit identity exposure as validated by the generation of a dynamic SD quality assessment report which is shared with the researchers. ConclusionThis SD pipeline ensures protected access to administrative datasets while upholding confidentiality. The automation of the framework streamlines workflow processes, minimises manual effort and enables timely and cost-effective access to SD by researchers. Future work could look at refining fidelity options and developing an R package for broader usability.

Ask AI

Helpful

Bookmark

View Full Paper