We designed an AI based tool to diagnose and help users write clear, accurate, and complete data descriptions. The tool's components include best practices data description guidelines, data descriptions reviewed by experts as few-shot prompts, and chain of thought reasoning to explain the diagnostic outputs. We engineered our prompts and Large Language Model choice so that a score of 8 reflects an acceptable data description. Users can double check the evaluations and the assisted descriptions to minimize scores inconsistent with expert reviewers and hallucinated outputs. The application is crafted to match the standards of our field and to be used with guided intention.
Nowacek et al. (Tue,) studied this question.