Weighted‐scoring scientific literacy instrument: development and validation of a contextualized assessment for junior high school students

Introduction: This study presents the development and validation of a contextualized scientific literacy assessment instrument for secondary school students, designed to overcome the limitations of traditional multiple-choice tests with dichotomous scoring. Weighted scoring allows for partial credit to be assigned to responses that demonstrate partial understanding, thus enabling a fairer evaluation. Methodology: A design-and-validation framework was employed to construct a multidimensional instrument based on PISA competencies and the Test of Scientific Literacy Skills, adapted to the national curriculum. Content validity was assessed by expert judgment, and the instrument was administered to 408 secondary students from 11 schools. Results: The Aiken’s V coefficients for content validity ranged from 0.73 to 0.87. Confirmatory factor analysis revealed strong loadings on a single factor (≥ 0.70; AVE = 0.665). After minor model adjustments, the fit indices were excellent (CFI = 0.999; RMSEA = 0.025), and internal consistency was high (composite reliability = 0.947). Discussion: These results indicate that the instrument meets statistical validity standards and supports the identification of key areas for improvement in scientific literacy. Conclusions: The instrument is valid and reliable for diagnosing levels of scientific literacy and for evaluating the effectiveness of educational interventions.