Visualizing Item Statistics for Better Developing Technology-Enhanced (TE) Items

Multiple-choice (MC) items have been used in educational testing for almost a century. The shift from paper-and-pencil to online administration has dramatically increased the popularity of technology-enhanced (TE) items relative to MC items. However, little research has examined how to use item statistics to develop high-quality TE items. The present study extended previous research on using item analysis for MC item development to TE item development. The proposed data visualization has been used in actual data review sessions. Feedback from subject matter experts (SMEs), along with item performance after the suggested revisions, demonstrates that the data visualization introduced in this paper is effective in helping SMEs identify problematic components of TE items. This, in turn, leads to more efficient and targeted revisions of TE items.