184 Developing quantitative writing evaluations for TL1 scientific writing programs

Objectives/Goals: Scientific writing is vital for TL1 trainees, yet evaluating writing programs effectiveness is challenging. Current evaluations gauge writing confidence, not skill competency. Our study aimed to develop a quantitative writing evaluation tool to assess technique and improvement longitudinally across the TL1 program. Methods/Study Population: The writing evaluation tool uses 19 typical grammar concepts embedded within three paragraphs. TL1 trainees received a list of these concepts without definitions and were instructed not to research them. They were asked to identify each concept, its location, and explain the reasoning behind of their choice (e.g., Qualifier: “very” and “significant” are redundant, thus “very” can be removed). Points were awarded when concepts were identified (0.5 point) and reasoned appropriately (0.5 point). The participants included 12 predoctoral and 3 postdoctoral trainees. Evaluations were administered at the start of Fall 2025 and again in Spring 2026. A paired t -test was used to assess longitudinal growth, with significance determined at p ≥ 0.05. Results/Anticipated Results: We separated our participants into two groups by experience level: 1Y) 8 trainees (1-year predoctoral) and 2Y) 4 (2-year predoctoral) and 3 (1-year postdoctoral) trainees. After the Fall 2025 evaluation, the highest score was 9.5, with the lowest score being 0.5. On average, group 1Y scored 3.8 and group 2Y scored 3.1, indicating that although the trainees were separated by experience level, their writing abilities did not differ ( p = 0.57). We anticipate that the Spring 2026 evaluation will show positive longitudinal growth for both groups as they learn to identify grammar concepts. Discussion/Significance of Impact: Developing an effective quantitative writing evaluation tool allows facilitators to refine writing programs for trainees. This study is the first step in testing the tool’s impact. Results will guide improvements in the evaluation tool and support ongoing evaluations of writing program effectiveness within TL1 training.