A Scalable Framework for Multidimensional Quality Assessment and Benchmarking of EEG Datasets

Ensuring the quality of datasets containing electroencephalography signals remains challenging due to heterogeneous acquisition protocols, diverse preprocessing practices, and inconsistent annotation standards. Existing toolboxes mainly focus on signal cleaning and pipeline checks, while generic data quality frameworks lack domain specificity and do not provide an interpretable dataset-level score for electroencephalography datasets used in tasks such as affect recognition and event-related paradigms. This work introduces a platform for dataset-level quality assessment of electroencephalography collections tailored to these downstream tasks. The platform computes a set of indicators that capture session completeness, consistency of session length, class balance, annotation timing, signal-to-noise ratio, outlier prevalence, feature information content, and class separability. It then combines these indicators into a normalized global Quality Score ranging from 0 to 100. The aggregation relies on order-preserving mappings and subject-wise summarization to enable robust comparisons across datasets and to prevent longer recordings from dominating the score. The evaluation uses six public datasets from affect recognition, event-related paradigm and motor imagery, all processed under identical protocols. Classification performance is reported using Balanced Accuracy and Area Under the ROC Curve. The global score reflects differences in data integrity, temporal consistency, and separability while remaining interpretable across heterogeneous collections. In addition, correlation analyses indicate positive associations between separability-oriented indicators and downstream performance, and negative associations with a higher prevalence of outliers. Targeted training adjustments guided by these indicators, including the removal of highly contaminated windows and loss reweighting based on class priors, consistently improve performance. Implementation and configuration files are publicly available to support reproducibility and adoption.