More Horizon/Charter Test Questions

After the allegations at Horizon Science Academy's Dayton campus were raised a couple weeks ago, I decided to look at Horizon's other schools to see if there was any evidence of test manipulation, similar to what was alleged to have gone on at Horizon's Dayton campus.

The way I looked at it was to compare the school's Performance Index (PI) score (it's state test results) with its ACT score (it's college entrance scores, which are handled through the College Board). The idea was that the school should score about the same on each. And, in fact, that's what happens in Ohio school districts. The correlation coefficient between a district's PI and ACT results is .79, which by most measures qualifies as a "strong" positive correlation, meaning the higher the PI, the higher the ACT, and vice versa. That .79 is about the same strength of correlation as exists between education level and income, just to give you a sense of scale and strength.

So in Ohio school districts, the state tests and ACT track pretty closely with one another.

Here's the pertinent info from the report I authored for Innovation Ohio:

"According to the latest State Report Card data, Horizon Science Academy Columbus scored a 98.6 on the state’s Performance Index (PI), placing it just below the state average score of 98.9 for all public school districts. The score was the highest of the four Horizon academies we reviewed. However, the average ACT score at Horizon Columbus was just 17, placing the students who took the test near the bottom in the 28th percentile."

Adding complication to the Horizon Columbus issue is that they are a National Blue Ribbon Award winning school. The Blue Ribbon school program was started in 1982 and recognizes the top performing schools in the country. The issue is that the award is based on test scores. If Horizon Science Academy Columbus fudged their scores, as is being alleged in Dayton and which their comparative test data suggest may be happening in Columbus, what does that do to the award?

And to go one step further, in Ohio's Charter Schools, those two data points are decidedly less strongly correlated, with a coefficient of .41. (The closer to 1 the correlation, the stronger the relationship) There remains a positive correlation, it's just much weaker in Ohio's Charter Schools. To be fair, there are only 18 of 281 Charter Schools on the Ohio report card that report having ACT scores, while all 612 school districts do. So with fewer datapoints, the Charter School data is more susceptible to being influenced by extreme examples.

However, Charters are all over the place on the two scores. Take the Arts and College Preparatory Academy in Franklin County. Their PI of 108.3 rivals the best Ohio school districts. Yet their 19 average ACT score (41st percentile nationally) is about 5 points lower than the 23.9 average ACT for the 10 school districts whose PI scores are between 108 and 109. The lowest district ACT score was 21 (55th percentile), the highest average ACT was 26 (84th percentile).

There were 543 school districts that scored better than Arts and College Prep on the average ACT, yet only 21 school districts scored higher on the PI. However, remove Arts and College Prep, and the correlation in Ohio's Charter Schools jumps to .46. Still not as strong as OHio's traditional public schools, but still stronger. And you can see what an influence extremes can have when you have fewer data points.

However, now that Auditor of State David Yost will be looking into the test disparities in the Horizon Columbus school as a result of our report, will he expand that to the other Charter Schools whose disparities are so striking?

As I made clear in our report, test disparity does not prove any funny business. But it sure merits examination by state regulators. And that should include traditional public schools too, by the way. However, the overall trend in traditional schools' scores suggests less funny business than the overall trend in Ohio's Charter Schools.

So stay tuned.