Improving the Methodology for Digital Analysis of the Consistency of Assessment Criteria Across Academic Courses in Higher Education Institutions

The article proposes a methodology for the digital analysis of the consistency of assessment criteria across academic courses in higher education institutions. The methodology integrates four components: a constructive alignment matrix linking learning outcomes, assessment criteria and levels of Bloom's taxonomy; the similarity of criterion weighting structures across courses, measured using cosine similarity and Jensen–Shannon distance; the comparability of grade distributions, assessed using standardised mean differences and the Kolmogorov–Smirnov statistic; and interrater agreement, measured using Krippendorff's alpha. These components are combined into a Criteria Consistency Index, which can be calculated using HEMIS-type records. Using a clearly identified simulated dataset covering eight courses, the index ranged from 69.7 to 93.5. Three courses were flagged for review, while one was identified as requiring recalibration of its assessment criteria. Validation using institutional HEMIS data was identified as the next stage of the research.