Calibrated uncertainty quantification for prosumer flexibility aggregation in ancillary service markets

Reliable forecasting of prosumer flexibility is critical for demand response aggregators participating in frequency controlled ancillary service markets, where reliability requirements such as the P90 standard are enforced. Limited behavioural data and dependence on exogenous forecasts introduce uncertainty into flexibility estimates, which can lead to reserve overestimation and financial penalties. This study applies and systematically evaluates established uncertainty quantification approaches based on Monte Carlo dropout (MCD), conformal prediction (CP), and conformalised quantile regression (CQR) to construct prediction intervals for aggregated prosumer flexibility and support uncertainty-aware aggregator bidding. The approaches are evaluated for a behind-the-meter aggregator participating in the Danish manual frequency restoration reserve capacity market. A large-scale synthetic dataset is generated using an industry-grade home energy management system, combined with publicly available load, solar, price, activation, and device-level data. Multiple multivariate CP strategies are evaluated using MCD predictions, alongside CQR using a separately trained quantile regression model. Under nominal synthetic test conditions, conformal methods substantially improve joint coverage compared with uncalibrated MCD baselines. More conservative methods satisfy the 90% lower-tail reliability requirement, while CQR and MCP provide comparatively narrow intervals and higher retained economic value. CQR and MCP retain approximately 69% and 68% of perfect-information profit, respectively, under the stricter penalty interpretation. A controlled robustness analysis further shows that increasing perturbations of load, solar, price, and activation inputs increase overbidding and reduce empirical coverage and economic performance. The results highlight the trade-off between reliability, economic value, and robustness to input uncertainty when applying established uncertainty quantification approaches to aggregator bidding.