Disaggregating behind-the-meter net load via fusing unsupervised learning and optimization

The rapid deployment of distributed photovoltaic systems behind the meter has created significant challenges for power system management, as the individual contributions of solar generation, baseline load, and line loss remain invisible to grid operators. Disaggregating the net load measured at distribution transformers into these components is therefore essential for accurate forecasting, planning, and operation. This paper proposes a novel generation-then-optimization framework that fuses conditional denoising diffusion probabilistic models with constrained capacity estimation for behind-the-meter net load disaggregation. In the offline stage, weather-conditioned diffusion models generate diverse and realistic photovoltaic and line loss pattern pools, while load patterns are estimated via statistical averaging. In the online stage, a pattern selection mechanism identifies the best-matching patterns from the pool, and a constrained optimization model estimates the capacity coefficients to recover each component. Comprehensive experiments on a one-year real-world substation dataset demonstrate that the proposed method achieves photovoltaic disaggregation with normalized mean absolute error below 0.65% and load disaggregation below 2.5% across five penetration levels (30%–150%). The conditional diffusion model outperforms variational autoencoders, generative adversarial networks, and TimeGAN in both generation diversity and fidelity, and the explicit inclusion of line loss provides structural regularization that significantly improves load estimation accuracy. The proposed framework is applicable to distribution networks worldwide with behind-the-meter solar penetration. • Conditional diffusion models generate diverse PV and line loss patterns. • A generation-then-optimization framework disaggregates net load. • Line loss modeling improves load disaggregation via bias reduction. • Pattern selection mechanism adapts to real-time weather conditions. • The framework is validated on one-year real-world substation data.