Evaluation criteria for AI-assisted product carbon footprinting systems: the cases of mapping and supply chain modeling

Large language models (LLMs) are increasingly used to automate components of product carbon footprinting (PCF). However, the carbon footprinting literature lacks criteria for evaluating the quality, documentation, uncertainty indication, and reviewability of AI-assisted systems for specific PCF use cases. We developed a set of evaluation criteria for AI-assisted carbon footprinting (AI-CF) through standards review, structured stakeholder elicitation with corporate practitioners and verification specialists, and iterative pilot testing. The resulting criteria are organized around transparent logic, benchmarking against expert judgment, and uncertainty characterization. The criteria were then applied to two use cases: automated life-cycle-inventory proxy mapping and cradle-to-gate PCF generation. Applying the criteria to two deployed proprietary AI-CF systems demonstrates that the framework is operational and can identify where human review or better data are most needed. For automated mapping, well-specified (“non-vague”) inputs achieved a 91.2% expert-accepted mapping rate, whereas inputs with ambiguous or incomplete descriptions (“vague”; e.g., internal product codes or generic descriptors) achieved 59.7%; repeat-run entropy increased with input ambiguity. For cradle-to-gate modeling, end-to-end benchmarking against 269 Environmental Product Declarations yielded a 33% median relative error, while component-level benchmark checks, though unvalidated, showed relatively strong pass rates for proxy mappings, hotspots, and direct emissions and weaker performance for key input rates. The framework provides a practical reference for AI-CF developers, users, and reviewers to assess AI-CF system and output quality for these use cases.