Meta: Error Detection in Data Quality for AI-Powered Business Intelligence

High data quality is crucial for reliable BI system insights. Yet, real-world datasets, like Historical Air Quality, often face issues such as missing values and inconsistencies, affecting predictive accuracy. This paper introduces an AI-based framework for error detection and quality assessment in BI systems, focusing on air quality data. It includes data ingestion, profiling, anomaly detection, and real-time feedback using ML models like Isolation Forests, Autoencoders, and LSTM networks. Splunk aids in real-time log monitoring and issue resolution. A data quality scoring mechanism improves insights and correction prioritization. Tested on the Historical Air Quality dataset, the framework improved predictive accuracy, with LSTM achieving an MAE of 6.12 and RMSE of 8.90, outperforming baselines like Bi-GRU and CNN-Bi-GRU. Data quality metrics improved by over 25% on average, highlighting the importance of robust data preprocessing and monitoring tools like Splunk in BI systems.