This thesis examines measurement validity in AI systems for constructs such as stigma and sound symbolism, indicating the need for more rigorous evaluation methods.
As AI systems are deployed in domains where the constructs they claim to measure (meaning, bias, cognition, social experience) cannot be observed directly, the gap between what evaluations claim to capture and what they actually capture has become difficult to audit. This thesis asks whether construct validity and reliability, drawn from measurement science, can be productively applied to two computational domains where this gap has gone largely unexamined. The first is sound symbolism. Using an adversarial scrubber over 810 size adjectives across 27 typologically diverse languages, I suppress genealogical signal while preserving semantic signal, and find a modest but consistent cross-family iconicity effect carried largely by vowel quality. The second is stigma in LLMs. Using the validated Individual Level Abortion Stigma Scale across 627 personas and five models, I show that representations of stigma fragment across cognitive, interpersonal, and structural levels. Together, the chapters show that rigorous measurement surfaces failure modes that standard evaluation cannot, motivating the need to ground auditing system performance in domain science, rather than generic benchmarks.
No takes yet. Share an insight, caveat, or question.
Anika Sharma (2026) studied this question.
Synapse has enriched 5 closely related papers on similar clinical questions. Consider them for comparative context: