Design of a Carbon Emission Monitoring and Prediction System Based on Big Data

Accurate monitoring and prediction of carbon emissions has become a key need by industries across the globe due to the growing need for sustainable development. Industrial activities generate large volumes of energy consumption data, creating opportunities for real-time monitoring and predictive analytics using big data technologies. A carbon emission monitoring and prediction systems is developed that integrates real-time electricity consumption data with external variables such as industry type, energy mix, and economic indicators. Big data platforms including Apache Hadoop and Apache Spark support large-scale data ingestion, storage, and processing. Carbon emission prediction is performed using a hybrid model that combines Long ShortTerm Memory (LSTM) networks for temporal pattern learning with Extreme Gradient Boosting (XGBoost) to capture feature-based relationships. Data preprocessing techniques such as normalization, feature engineering, and missing value imputation improve data quality and model reliability. The dataset consists of large-scale industrial energy consumption and carbon emission records collected at an hourly resolution, comprising approximately 10,000 samples. The data is divided into training and testing sets using an 80:20 split. The LSTM model is configured with two layers and 128 hidden units, using a learning rate of 0.001 with the Adam optimizer. The XGBoost model employs 100 estimators with a maximum depth of 6 and regularization parameters ( λ = 1.0, γ = 0.1 ). Experimental evaluation shows that the hybrid LSTM-XGBoost model outperforms alternative approaches including CNN-LSTM-BERT and AdaBoost, achieving MAE = 0.0234, MSE = 0.00086, and R^2 = 0.963. The framework supports real-time carbon emission monitoring and forecasting, providing a reliable tool for data-driven industrial emission management and sustainable decision-making.