Development of Level-Specific Tests Using the CEFR-J Listening Descriptors

While the use of CEFR-J, a localized version of the CEFR for the Japanese contexts (Tono & Negishi, 2012), can provide firm bases for program development and test design, researchers (e.g., Fulchur, 2010;Runnels, 2013) have expressed concerns regarding its illustrative nature and the absence of an underlying psycholinguistic theory.In this study, English listening tests were developed based on the CEFR-J and were administered to 217 English learners.IRT item analyses and the Bayesian hypothesis testing were conducted to examine: 1) if the rank-ordering of the carefully constructed test items is pertinent to their intended levels of A1.2 through B2.1, and 2) the use of the CEFR-J listening scales helps to develop level specific tests with the systematic increase of the mean difficulty from low to high levels.The results indicate that the items rank-ordered based on their difficulty parameters demonstrated an implicational progression from A1.2 to B2.1; however, when the logit means of the sub-levels were considered, the distinction between A2.2 and B1.1 was not clear.Finally, it was confirmed that the development of a level-specific listening test based on the CEFR-J may be feasible when the development procedures are carefully coordinated.