The Rasch Analysis of Item Response Theory: An Untouched Area in Evaluating Student Academic Translations

Classical test theory (CTT) has been widely applied to analyze objective test data. CTT utilizes aggregated data and descriptive approaches to contribute to the detection and elimination of measurement error sources and a test’s unreliability. On the other hand, the Rasch analysis of item response theory (IRT) employs complex methods and scales to generate outputs that identify measurement error sources and a test’s unreliability. Additionally, it is used to examine the interaction between an item’s difficulty and students’ ability (in our case, students’ translation competence). In this regard, eighteen multiple-choice translation questions were thoroughly analyzed through the Rasch analysis to substantiate the beneficial outputs that can be maintained. The application of IRT provides a deeper insight considering “a range of information on the behavior of individual test items and individual students as well as the underlying construct being examined” (Tavakol & Dennick 2013: 838). To evaluate item facility/difficulty across students’ competence, visual displays such as item characteristic curves (ICCs) and item difficulty stability (item parameter invariance) were used to identify how well an item’s difficulty corresponded to students’ competence. In addition, other Rasch premises and characteristics such as a test’s unidimensionality, local independence, and Rasch fit statistics (infit and outfit) were analyzed to identify how well a translation test only measured one underlying construct, how well items were independent of one another, and how well items fit the model, respectively. The limitations and implications were also discussed.