Testing the Boundaries: A Series on Ohio (and the Nation's) Achievement Gap. Part I: After 12 years, Ohio's Performance Disparity Between Rich and Poor Districts has Grown Worse. Now What?

One the worst traps we can fall in as analysts is to ignore the big picture. Lately, I've been guilty of this -- dealing with short term, juicy topics while eschewing the forest in front of my face.

So I decided to look at 12 years of Ohio Graduation Test data and see how different districts fared each year. The OGT results are listed separately on the Ohio Department of Education website.

The results blew me away; after more than a decade of test-focused reform, Ohio's achievement gap between its wealthiest and poorest districts has gotten worse, not better. What now? Well, I'm going to do my part: namely a blog series on exactly that: How do we determine academic success and why aren't we closing the gap?

The data that inspired this series comes from the OGT data put out by the Ohio Department of Education. I chose OGT data because that's been the least volatile state-administered test. ODE lists the percentage of students who scored advanced, accelerated, proficient, basic and limited for each year's testing. Then, using the state's Performance Index formula (I didn't include the new "Advanced Plus" category for the calculus. Again, for all you nerds out there.), I was able to crunch those four categories into a single, mini-Performance Index (PI) Score so I could more easily see how districts were improving.

I then looked at the districts' improvement on the raw mini-PI score and how they ranked each year among the 608 districts that could be compared each year. I didn't include the island districts or College Corner.

Then I looked at their typeology. While the typeology numbers and definitions have changed slightly over the years, the typeology tells you the kind of district based on community make up and poverty. Here's the most current typeology chart:

The typeology make up is interesting in and of itself. For example, you can see that about 2/3 of Ohio's school districts are in small towns or rural communities. Yet 2/3 of Ohio's school kids are in suburban and urban districts.

This explains Ohio's struggle with school funding to a great extent. Because ways you can make a formula work for rural districts will likely hurt suburban and urban areas, where most of the kids are.

But I digress.

Again, I used the typeology chart to determine which typeologies tended to score better than others in each year. They I looked to see how they improved (or didn't) between the 03-04 school year and the 14-15 school year.

The results aren't really surprising. The wealthiest categories (3,5 and 6) rated the best. The poorest (categories 1, 4, 7, and 8) did the worst. What is surprising is this: The achievement gap between the rich and poor districts is growing more pronounced after a dozen years of test obsession.

For example, on the 2003-2004 Math OGT, category 6 (very wealthy, suburban districts like Ottawa Hills) made up 46.7% of the top 10% mini-PI scores. In 2014-2015, that had jumped about 10 percentage points to 56.3%. Meanwhile, the poorest urban districts (category 7, which includes districts like Euclid and category 8) made up 38.3% of the bottom 10% scoring districts, and 6 of the 8 Big 8 urbans (category 8, which is Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo and Youngstown) scored in the bottom 10%. On the 2014-2015 OGT Math, all the Big 8 urban districts score in the bottom 10% and 56.6% of the bottom 10% of scores come from the state's poorest urban districts.

The same general breakdown and change has occurred on reading scores, though not quite as dramatically. But the disparity still has grown significantly between the wealthiest and poorest districts.

So even though urban districts only make up about 9% of all districts in the state, they make up nearly 60% of the 60 lowest scoring districts in the state. Likewise, the state's wealthiest suburban districts (category 6) account for 7.6% of all Ohio districts, but make up about the same 60% of the 60 highest performing districts.

It is equally telling that in neither 2003-2004, nor in 2014-2015 did a single urban district score in the top 10% on either OGT category. And only 1 wealthy suburban district scored in the bottom 10% of either test in either year (Gahanna on 2003-2004 Reading).

What does all this mean? Well, it appears that, generally speaking, 12 years of test-focused accountability has grown the achievement gap between the state's wealthiest and poorest districts, not shrunk it. But I want to ask a different question: Can this disparity ever shrink?

We've known for years about the powerful connection between test scores and poverty. And we've tried to mitigate the problem by using value-added scores, or some other statistical pretzel, but the fact remains that the data produced by test scores has as much (if not more) to do with poverty than classroom performance. 

This calls into question our whole test-based accountability scheme. For example, if no Big 8 Urban district scored outside the bottom 10% on this set of OGT scores, is it a failure of the Big 8, or merely a confirmation that the tax and census data showing the extreme poverty in these communities is accurate? And if that is so, is it fair to hold these districts, buildings, teachers and even communities liable for the district's performance?

And if these scores are measuring poverty rather than quality, should we be opening the doors to more and often poorer performing choice options in these districts?

And if test scores aren't cutting the mustard, what can? And at what cost?

These are all questions I'll be exploring over the next several days as I dig into this series. But I think it's important to recognize that the state and nation's poverty achievement gap, if we keep measuring it through tests, may never close. We may only get a true assessment of our nation's education system when we stop using subject-based standardized tests to measure achievement.

Monday: Testing the Boundaries
Part II - What Can Outliers Teach Us?

Federal Data: Ohio Charter Schools Widen Ohio's Already Too Wide Achievement Gap

Recently, the White House put out a report outlining the country's student achievement gap, and the news wasn’t great for Ohio. 

We had the nation’s ninth largest reading gap between our highest and lowest performing schools, the second-largest math achievement gap, and the fourth largest graduation gap. While much of this difference can be explained by the high performance of our highest performing schools, the gap is and should be a serious concern for Ohio’s educators, parents and policy makers.

What the data show, however, is that far from being a solution to the achievement gap issue here, Ohio's charter schools are part of the problem.

I wrote about this issue at Innovation Ohio last night.

Here are what the data tell us:

• Despite making up 8% of all Ohio school buildings, charters represent 13% of the worst-performing math buildings, 31% of the worst-performing reading buildings, and 78% of the buildings with the worst graduation rates.
• Ohio’s achievement gap is 6% bigger in math, 8% bigger in reading and a whopping 23% bigger in graduation rates than they would be if the analysis included just local public schools.
• And while the state’s achievement gap is still too large, in all three cases, eliminating charters from the calculation drops Ohio’s achievement gap ranking. Math drops from second to fourth greatest. Reading falls from ninth to 11th greatest. And the state’s graduation rate gap tumbles from fourth to 14th highest.
• The achievement gap is greater in charter schools for math than it is in the local public schools
• The charter school achievement gap is narrower in reading and graduation rates because charters’ highest performers are so low performing overall compared with local public schools. For example, the average graduation rate for the 19 highest-performing charters – defined as those that have greater than 60% graduation rates – is 65%. Those 19 charters represent 17% of eligible charters. The average rate for the highest performing local schools – 96% of which have graduation rates greater than the 60% threshold – is 91%.

There is work that needs done closing Ohio’s achievement gaps in all schools, no question. But what the federal data clearly show is that charter schools don’t provide an overall solution. In fact, they are part of the problem -- especially on graduation rates.

Folks in Youngstown and other places should take note of this federal data: Relying on charter schools to close achievement gaps in Ohio has not worked. In fact, it has led to greater gaps in student achievement overall. So before the new CEO in Youngstown decides to turn all of that city's schools into charters or something, here's hoping he or she looks at the evidence first and carefully considers district options.

As for the gaps themselves, much of Ohio’s gap problem is driven by our highest scoring local public buildings scoring so well. For example, while our lowest-performing math buildings score an average proficiency rate of 26% – the same as West Virginia – our remaining buildings score a 78% – the nation’s seventh-highest rate and far higher than West Virginia’s 47% – the nation’s fourth-worst showing. So while West Virginia’s gap seems to be much narrower, it’s because the state’s schools perform so much worse overall than Ohio’s do.

So, if there’s a silver lining to the achievement gap report it’s that 95% of our schools are doing a pretty good job. However, we must address the 5% that are struggling mightily by utilizing – and paying for – measures that research shows can help improve student achievement.

Here is how each type of school does in Ohio, with the White House figures reported on the top line. The numbers are percentages. In reading and math, it's measuring proficiency. In graduation, it's measuring the graduation rate.