Validating the Prompt Engineering Expertise Battery (PEEB) as a Combined Self-Report and Performance-Based Measure for Generative AI

Generative AI is used through natural-language prompting, yet few validated measures test whether self-assessed prompting competence matches demonstrated skill. We developed the Prompt Engineering Expertise Battery (PEEB), a 29-item self-report scale and a five-task performance module organized around seven dimensions. In Phase 1 (N = 257), internal consistency was high (α = 0.97), but exploratory parallel analysis retained a single factor, which replicated across random split halves; the scale is therefore scored as one total, captured by a 14-item short form (α = 0.95). In Phase 2, 60 participants completed screen-shared tasks scored by three raters (ICC(2,k) = 0.94). Self-report related only weakly to performance (r = 0.18, 95% CI [−0.07, 0.42]), a preliminary estimate; prompts were brief and disclosed sensitive information in 71% of health tasks. Assessments of prompting competence should include task-based measures, and the structure, short form, and performance module need confirmation in an independent sample.