LLM-Assisted and Rule-Based Assessment of ESG Disclosure Quality and Its Association with External ESG Ratings: Exploratory Evidence from S&P 500 Energy Firms

Environmental, social, and governance (ESG) disclosure has become an important source of information for external stakeholders. As sustainability reporting has expanded, distinguishing disclosure quantity from disclosure quality has become increasingly important. This study examines how sustainability disclosure quality is associated with external ESG evaluation outcomes among Standard & Poor’s (S&P) 500 Energy Sector firms. ESG-related claims were identified and classified from sustainability reports using large language model (LLM)-assisted structured content analysis. Based on the resulting corpus, three disclosure quality indicators were constructed: the Quantitative Evidence Ratio (QER), the Target Accountability Ratio (TAR), and the Reporting Infrastructure Score (RIS). These indicators were integrated into a composite Disclosure Quality Index (DQI) and examined in relation to S&P Global ESG Scores using Spearman’s rank correlation. The results indicated limited positive associations for the individual indicators, whereas the composite DQI showed a more pronounced positive relationship. Disclosure–rating divergence patterns were also observed, indicating that relatively favorable disclosure quality positions do not consistently correspond to higher ESG Score rankings. Overall, the findings suggested that sustainability disclosure quality may be multidimensional and that its association with external ESG evaluation outcomes became more apparent when disclosure characteristics were considered in combination. However, because the analysis is restricted to a small sample of S&P 500 Energy Sector firms, the findings should be interpreted as exploratory sector-specific evidence with limited generalizability.