Development of Level-Specific Tests Using the CEFR-J Listening Descriptors

While the use of CEFR-J, a localized version of the CEFR for the Japanese contexts (Tono Runnels, 2013) have expressed concerns regarding its illustrative nature and the absence of an underlying psycholinguistic theory.In this study, English listening tests were developed based on the CEFR-J and were administered to 217 English learners.IRT item analyses and the Bayesian hypothesis testing were conducted to examine: 1) if the rank-ordering of the carefully constructed test items is pertinent to their intended levels of A1.2 through B2.1, and 2) the use of the CEFR-J listening scales helps to develop level specific tests with the systematic increase of the mean difficulty from low to high levels.The results indicate that the items rank-ordered based on their difficulty parameters demonstrated an implicational progression from A1.2 to B2.1; however, when the logit means of the sub-levels were considered, the distinction between A2.2 and B1.1 was not clear.Finally, it was confirmed that the development of a level-specific listening test based on the CEFR-J may be feasible when the development procedures are carefully coordinated.