Answers are grounded in this paper and cite their sources.
Targeted evaluation platform enhances access and standardization in syntactic language model assessments.
Targeted syntactic evaluations have yielded insights into the generalizations learned by neural network language models. However, this line of research requires an uncommon confluence of skills: both the theoretical knowledge needed to design controlled psycholinguistic experiments, and the technical proficiency needed to train and deploy large-scale language models. We present SyntaxGym, an online platform designed to make targeted evaluations accessible to both experts in NLP and linguistics, reproducible across computing environments, and standardized following the norms of psycholinguistic experimental design. This paper releases two tools of independent value for the computational linguistics community:
No takes yet. Share an insight, caveat, or question.
Gauthier et al. (2020) studied this question.
Synapse has enriched 8 closely related papers on similar clinical questions. Consider them for comparative context: