The Use of Ninth-Grade Early Warning Indicators to Improve Chicago Schools

Abstract Chicago has been in the forefront of the country in its use of 9th-grade indicators of dropout. Catalyzed by the development of the freshman on-track indicator and research around it, Chicago school administrators, central office personnel, and external partners have developed a number of mechanisms using 9th-grade indicators to stimulate school improvement. This article describes 3 ways in which early warning indicators are useful for improving student achievement: (a) focusing conversations and efforts on actionable problems; (b) identifying students for intervention; and (c) using indicator patterns to address low performance in a strategic way. Examples from high schools in Chicago suggest that knowledge of the on-track indicator and its use in district accountability were not enough to change practice. However, the availability of data tools that make it easy to act on information about on-track rates have changed the ways in which teachers and school staff interact with each other, students, and parents regarding improving student performance. The strategies they have developed with the data tools have provided a systematic focus to their efforts, which appears to be paying off in substantially improved ninth-grade achievement. ACKNOWLEDGEMENTS I acknowledge my colleagues Melissa Roderick and John Easton, who drove much of the work at CCSR around the on-track indicator. My colleagues Melissa Roderick and John Easton drove much of the work at CCSR around the on-track indicator. Jenny Nagaoka and David Stevens provided very helpful feedback on this article, and Todd Rosenkranz, Jessica Puller, and Marisa de la Torre provided technical assistance. Nick Montgomery collaborated on the presentation that generated this article. David Stevens, Amber Stitziel Pareja, David Johnson, Marisa de la Torre, Todd Rosenkranz, and Desmond Patton worked with me on the Focus on Freshman study of the transition to high school. CCSR is very grateful to the Spencer Foundation for supporting work of CCSR around high school graduation, dropout, and on-track rates. Notes 1The indicator had been developed by Miller, Luppescu, and Correa (Citation2003) as a means of providing feedback to middle schools about how their graduates performed when they entered high school (an example is available at http://ccsr.uchicago.edu/publications/how-well-do-vivaldi-students-succeed-after-elementary-school). Notes. Predictions were calculated through logistic regression models. Variables included in the model under "background characteristics" were: gender, race (African American, Latino, White, Asian, American Indian), mobility measured as the number of times student switched schools in the 3 years prior to high school, eighth grade math score on the Iowa Tests of Basic Skills (ITBS), eighth-grade reading score on the ITBS, months old-for-grade when began school, began school young for grade, began school old for grade. In addition, there were several indicators of economic status derived by matching students' residential address to information from the US Census on the block group in which they live, including the percentage of men unemployed, the percentage of families under the poverty line, median family income, and average years of education. 2From Allensworth and Easton (Citation2007). Predictions were calculated through logistic regression models from which overall correct prediction and specificity (predicting nongraduates) are included. Variables included in the model were: gender, race (African American, Latino, White, Asian, American Indian), mobility measured as the number of times student switched schools in the 3 years prior to high school, eighth-grade math score on the Iowa Tests of Basic Skills (ITBS), eighth-grade reading score on the ITBS, months old-for-grade when began school, began school young for grade, began school old for grade. In addition, several indicators of socioeconomic status were derived by matching students' residential addresses to information from the US Census on the block group in which they live, including the percentage of men unemployed, the percentage of families under the poverty line, median family income, and average years of education. 3From Allensworth and Easton (Citation2007). Variance explained comes from the R-square statistic from regression models predicting percentage of semester courses failed. Background variables are the same as described in note 2. Attendance is measured in days, with course cutting counted as partial days (e.g., one course missed out of seven counts as 1/7 of a day of absence). Study habits are measured through student surveys that ask: (a) I set aside time to do my homework and study; (b) I try to do well on my schoolwork even when it isn't interesting to me; (c) If I need to study, I don't go out with my friends; and (d) I always study for tests. 4This research is in process. The research team included David Stevens, Elaine Allensworth, Amber Stitziel Pareja, David Johnson, Marisa de la Torre, Todd Rosenkranz, Melissa Roderick, and Desmond Patton. 5The study measure consists of four items: (a) I set aside time to do my homework and study; (b) I try to do well on my schoolwork even when it isn't interesting to me; (c) If I need to study, I don't go out with my friends; and (d) I always study for tests. CCSR surveys are given to all CPS students in grades 6–10 and have been administered every other year since 1997. As a result, one can compare students' responses to the surveys over time. The decline of .21 standard deviations is derived from a nested model where level one is a measurement model, level two is observations, and level three is students. Variables are entered at the observation level representing students' grade (six through 12), with coefficients representing the difference from ninth grade. The model also has controls for year of survey administration at the observation level and student characteristics at the student level. Student characteristics include gender, race, old for grade, special education status, limited English proficient status, free or reduced-priced lunch, and students' eighth-grade reading score. The survey analysis was based on all respondents to the districtwide surveys, which is over 100,000 students in each survey year in grades 6–12. 6See note 4 for a description of the models that were used to calculate change from the middle grades to ninth grade. Questions that went into the measure of monitoring and support include: My teacher: Notices if I have trouble learning something; Teacher really listens to what I have to say; Helps me catch up if I am behind; Will help me improve my work if I do poorly on an assignment; Gives me specific suggestions about how I can improve my work in this class; Explains things in a different way if I do not understand something in class; Is willing to give extra help on schoolwork if I need it.. 7We compared average student grades across schools, controlling for students' entering characteristics. Hierarchical models predicting ninth-grade grades were run with students nested in classrooms, nested in schools. At the student level, controls were included for students' eighth-grade test scores in reading and math, and their English and math scores on the EXPLORE test taken in the fall of ninth grade, race, gender, socioeconomic status, age, and mobility. At the classroom level, control variables were included for whether the class was in the fall or the spring semester, average incoming test scores of students in the class, and type of class. In one school with better-than-expected attendance and grades (significantly positive school residuals), an on-track lab coordinator closely monitored students' grades and called them in for conferences. This school also made use of the data reports in ninth-grade teacher teams. At another school with better-than-expected performance, course attendance was tightly enforced by all staff in the school, with teachers calling home every day that a student missed their class. 8These are available at http://ccsr.uchicago.edu/publications/what-matters-staying-track-and-graduating-chicago-public-schools. 9These reports are available at http://ccsr.uchicago.edu/school, under Getting On-Track: How Your School is Doing with the Freshman Year. CCSR director Melissa Roderick worked with schools in the Network, and with researchers at CCSR, in an iterative process of report development and refinement. 10As the first in a three-part series of exams culminating in the ACT, the ACT Explore exam is designed to be administered to eighth and ninth graders (followed by the PLAN in tenth grade) to assess their academic skills in English, reading, math, and science. 11ISAT is the state test that measures reading and mathematics achievement in grades 3 through 8 and science achievement in grades 4 and 7.