How Do We Know The New State Report Card is More Accurate?

Much was made yesterday of the Fordham Institute's statement from Aaron Churchill, whom I respect and admire, that said the statewide drop in grades on the state's new report card was how students "actually performed" last year.

Beyond the obvious logical issue (weren't previous years' tests also measuring how kids "actually performed" on those tests?), Aaron Churchill's point was that the state's been glossing over the struggles kids have been having for far too long. And now that we have a rigorous test, we will be seeing how kids "really" perform.

I'm sorry, but I just don't see it. And the simple reason is this: Just because kids do worse on this set of tests doesn't mean the old ones were soft or something. How do we know this new round is more accurate? Because test scores dropped?

Could it be that the new tests were bad? If I develop a test designed to get only 1/3 of my students to pass, guess what? It's a bad test and I would be forced to re-write it. I'm not saying the PARCC tests fit that description, but you can't just say the tests are more accurate because kids do worse on them. Unless you're predisposed to think that public schools are failing.

That predisposition reveals the education reform movement's central fallacy: Public Schools aren't doing the job. So advocates become blind to success, or they insist that success means assessments are too easy or something.

Why can't success be, well, success?

I just grow weary of accountability advocates telling us to adopt a rigorous regime, then when kids start doing well at that regime, they tell us that old regime is feeble so we need to adopt an even more rigorous one, which (guess what?) kids will master in a few years too. But for a time, at least, they look like they're not doing as well.

Seems like a self-fulfilling prophecy to me.

I am not against assessments. I think we need to find out how our kids are doing. In fact, educators have been testing and grading kids since, well, forever. The difference today is that these assessments are now being used to do far more than they were ever intended. They drive funding. They drive school closure. They drive levy success. They drive everything.

I'm not even opposed to the Common Core, per se. I like the concepts my kids are learning in math and wish I had learned math that way. I also think it's a good idea to ensure that if a kid moves to Ohio from Tennessee, teachers won't have to catch up the kid for 6 months. My problem is putting so much emphasis on these test results that whole communities suffer and neighborhoods lose their core.

After all, it was none other than Thomas Jefferson who set aside the heart of every American community for "public schools" in the Land Ordinance of 1785.

Assessments need to look at more than analytical ability. That's important, but it's not everything. Creativity, practicality, intuition, critical thinking, love of learning, curiosity, etc. are all as important, if not more so. Yet we do not assess those attributes at all.

This new report card data won't probably end up meaning much because the tests coming in this year are different and there are so many opt-outs, appeals and incomplete data. So I don't know how valuable this set will be beyond this year. I think it's problematic that more than 20% of school district grades are Fs when districts never earned more than single digits in previous years.

Were the single digit years more accurate, or the 20%?

I have no idea. And you know what?

No one else knows either.

Unless you're predisposed to think public schools can't be doing as well as their past test scores suggest, or that public schools are flawless or something. And if that's the case, then you've got a bias that could be blinding you to what could be some great news: that our public schools are doing a great job for the most part, or some bad news: that they need help. Yes, there remain struggles serving at-risk kids. Yes poor kids have less chance of doing well on these tests than wealthy ones. But how do you separate these issues from the tests being inherently tied to demographics?

Overall, I'm damn proud to live in a country whose founders believed the beat to every community's heart started in the classroom. And for those who think our public schools suck compared to anywhere else in the world, look at our economic strength, our creativity, our innovation and compare it with any high-scoring country on international tests (many of the highest-scoring countries 20-30 years ago are in economic chaos today).

And ask a very simple question: Where else would you like to live?

I think America has always has been great. And it is our commitment to Jefferson's heart -- our public schools -- that has played among the most important roles creating that greatness.

Don't let anyone tell you different.

New Report Card: Greater Percentage of Urban Buildings Meet High-Quality Charter Definition Than Charters

There has been lots of discussion recently about how many charter schools meet the state's definition of a "high-quality" school. The primary state "high-quality" definition is a school with a student growth grade of A or B on the state report card and a performance index (proficiency) grade of C or higher. There are a couple other types of definitions (allowing a charter to receive a B or higher on student growth while showing three years of improvement on performance index, even if it's below a C). But if we want to look at the most exclusive definition, it is the A or B in growth and at least a C in performance index.

Even though about 1/2 of charter students do not come from the state's Big 8 Urban districts (Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo, Youngstown), I thought it might be illuminating to use the new state report cards to see how many charters and how many Big 8 urban buildings meet that most exclusive definition.

The results are fascinating.

There are 33 Big 8 Urban buildings that meet that definition, representing 8% of all Big 8 buildings. Those 33 buildings do not include any district-sponsored charter schools.  Meanwhile, only 14 of 288 charters (including district sponsored ones) that receive report card grades would qualify. That's a rate of 5%. There are about 100 more charters in Ohio, but they are evaluated under a different system.

So the state's urban districts produce high-quality buildings at a 60% greater rate than the state's charter schools, which only have about 1/2 of their students coming from those urban districts.

And here's how the highest performing buildings in each sector perform:

High-quality urban buildings produce 60% As and Bs on the report card. High-quality charters produce 55% As and Bs. But it is on the other end of the scale where the performance difference is most profound. That's because while 16.6% of high-quality urban building grades are Ds and Fs, an astonishing 28% of high-quality charter grades are Ds and Fs.

What this is meant to show is something pretty simple: While high-performing charters are out there and are doing amazing things, there are even more high-performing Big 8 Urban buildings that are also doing amazing things. And it's time we recognize their accomplishments as often and vigorously as we're willing to correctly acknowledge high-performing charter school accomplishments.

And while only having 8% of Big 8 buildings meet this most exclusive "high-quality" definition is still far too infrequent, the fact is it's about a 60% more frequent occurrence than in Ohio's charter schools.

What else is interesting that those 33 high-quality buildings come from 5 of the Big 8 districts.

District	Number of High-Quality Buildings
Akron	9
Canton	2
Cincinnati	13
Cleveland	4
Columbus	5

So while it's encouraging that kids in most of Ohio's urban districts do have some high-quality options, there remain too few in too few districts.

New Report Card: Charters Perform Slightly Better in 2 of 9 Categories, Slightly Worse in 1 and Far Worse in 6

Charter Schools have slightly higher percentages of As and Bs in one report card grade category and slightly lower percentages of Ds and Fs than districts in that same report card category, plus another this year. However, districts clobber charters in every other comparable category. By a lot.

The two report card categories where charters outperform districts are in student growth among the lowest performing students and growth among special needs students. In the lowest performing students category, charters had 42.2% Ds and Fs and 27.1% As and Bs. Districts had 49.8% Ds and Fs and 26.7% As and Bs. Meanwhile, in the special needs category, charters had 45.2% Ds and Fs and 24.6% As and Bs. Likewise, districts had 52.1% Ds and Fs and 27.1% As and Bs.

However, the other categories were striking for how much districts outperformed charters. In 5 of the remaining 7 categories, more than 80% of charter grades were D or F. In one other, 70% were Ds and Fs. In Indicators Met and closing achievement gaps (AMO), more than 90% of charter grades were D or F. By contrast, more than 1/2 of district grades in Indicators Met were As or Bs and more than 30% were As or Bs in closing achievement gaps.

The only category where charter performance was even remotely close to districts' was in overall value added, or student growth. While 52.7% of charter grades were D or F and 31.1% were A or B, 48.5% of district grades were D or F and 40.1% were A or B.

So while I expressed concern in my previous post about districts' ability to address the needs of our state's most needy, those concerns are overwhelmed by our state's charter schools' stunning inability to perform across so many report card categories. And even where they do perform better than districts, it's only slightly better.

Let me bottom line Ohio's charter school performance: For a public investment of what is now $1 billion a year, along with 16 years of commitment, we have a charter system that provides slightly better performance than districts in two report card categories, slightly worse in one other and dramatically worse in 6 others.

Remember that charters take about 1/2 their students from districts outside Ohio's urban core.

That's not a performance record one would expect given this state's three-decade policy and financial commitment. Let's hope the state's new found quality focus will change this story. And soon.

New Ohio Report Card: Too Many Districts Struggle With Most Needy

While the new Ohio report cards continue to show the overall trend of Ohio's public school districts doing a better job overall than Ohio's charter schools, there is troubling data indicating that our districts struggle to improve student achievement for the our most needy students.

When it comes to serving the lowest-performing students, students with disabilities and closing achievement gaps (AMO), Ohio's school districts have about 40% Fs.

And even on student growth, it's feast or famine. The highest percentages for overall value added (student growth) are Fs and As. The disparity is quite stark, with more than 80% of all grades given to school districts being As or Fs. I'm not certain this is in fact the case (I always am suspect about these either-or results), but the data are not encouraging. 

Where districts do very well is graduating students. About 1/2 of all grades given on graduation rates for both 4 and 5 year cohorts are As. That's good.

In addition, there were no Fs given for Performance Index scores -- the amalgamated number created to indicate a district's proficiency success. The percentages of As and Bs are down from previous years, though I think that's a function of the PARCC exam slip everyone suspected would happen. But it didn't cause as much of a drop as other states' experiences.

Again, I caution about reading too much into these data because this year's report cards are fraught with opt-outs, safe harbors, tests we aren't going to be using anymore and incomplete ratings in some categories. And these conclusions are based on the extremely limited testing regime currently used. 

But these data should cause concern that we are not serving kids who need the most help in enough places.

And while I certainly believe that cutting funding for education, as this governor and legislature have done over the last several years hasn't helped districts cope with the needs of their most at-risk kids (for it tends to be the districts with the most need that have the least ability to self fund their interventions), we must confront these issues as a state and community.

It will indeed be interesting to see how Ohio's charter schools fare in these categories overall -- an analysis I will be performing next.

CORRECTED New Ohio Report Cards: Urban Districts Outperform Charters. But Only 1/2 of Charter Kids Come From There

There's some good news (or bad news, depending on how you look at it) for charter schools in the new state report cards. Charters have slightly lower percentages of Ds and Fs on the state report card than the Big 8 urban districts (Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo and Youngstown).

And charters still have 73% Ds and Fs. The Big 8 has 79% Ds and Fs. However, Big 8 urban districts have a slightly higher rate of As and Bs with 14.5% to charters' 14%. Where charters really outperform urbans is on C grades. Urbans barely have any of those grades.

Here's the problem, though. Barely 1/2 of kids in Ohio charter schools come from those districts anymore.

When charters are compared with Ohio school districts overall, they woefully underperform the districts from which nearly 1/2 of their student population originates. And these data don't include the state's worst-performing charter schools -- the Dropout Recovery schools. Those schools have their own evaluation, which is woefully weak, as I pointed out last month.

I caution anyone against reading too much into this year's data for a very simple reason: it probably won't mean much. Many more kids than usual opted out of the PARCC exams last year, upon which many of these grades are based. In addition, schools are receiving safe harbor from these grades because the state switched over from PARCC. Not only that, but many schools and districts are appealing these grades. So they very well could change.

But there are bits of good news here. For instance, when some states switched to the PARCC exams, their grades dropped precipitously. While Ohio's grades did drop, they didn't do so ridiculously by claiming, for example, that only 1/3 of Ohio's kids could read or something.

I will continue diving deeper into these data this afternoon. But overall, these data are telling the same story we've heard now for 16 years -- kids in Ohio's school districts overall are performing much better on state assessments than students in Ohio's charter schools, especially considering that 1/2 of Ohio charter school kids now come from outside the state's urban core.

CORRECTED New Ohio Report Cards: More than 1/2 of all Charter Grades are Fs. District Grades Drop Too.

I will be posting more details throughout the day on the new state report card data the Ohio Department of Education just released. But the general trends aren't great, especially for charter schools, but also for districts. Then again, it's tough to judge this year's results because they were based on tests we don't administer any more and there are protests, appeals, opt outs and other complications that make dealing with this year's report card uniquely troublesome.

But overall, 53% of charter school grades released today are Fs. More than 70% are Ds and Fs. Meanwhile, more than 60% of district grades are A, B, and C. Though more Fs were given to districts than at any time in the past, with about 21% of district grades being Fs now. Though more than 40% of all district grades are As and Bs. Meanwhile, less than 15% of charter grades are As or Bs.

I need to combine the grades issued last month with these grades to get an overall report card status, but I believe the percentage of Fs will increase for charters after graduation rate grades are factored, while I think the percentage of Fs will drop for districts once those data are included. But stay tuned.

Here's the percentage of grade distribution for the grades released today:

New Proper Perspective: Whither $71 million?

The Proper Perspective: An honest exchange about Ohio’s Charter School Program grant

Ohio won a $71 million federal Charter School Program (CSP) grant last fall, but after backlash about the original grant application (which described Ohio as a beacon of charter oversight and overstated the performance of the charter sector), the U.S. Department of Education put a hold on the money. Ohio’s latest response to the feds was on January 29. Jamie and Steve have both been writing on the topic recently and had an exchange about the fate of the grant.

*****************

 Jamie: Ohio has lots of high-performing charter schools that can benefit from CSP, and I take issue with you suggesting that the feds take back the money and dole it out elsewhere.

Steve: I’m in no way suggesting the feds take back the money. All I’m saying is I understand if they do because the Ohio Department of Education so misled them on the initial grant application. And if they do take it back, we have only the adults at ODE to blame. Surely you’re not suggesting that it was an honest assessment of the Ohio charter school sector for the department to claim that zero charters were poor performing in 2012–13 and only six were in 2013–14?

Jamie: That's good, because you did seem to suggest it—as well as suggesting that there weren't enough high-performing charter schools in Ohio to expand their footprint (you and I both know better). Students shouldn't have to bear the responsibility for errors committed by the department. To your question: No, I don't think that was a fair assessment. The original application used a "federal" definition of failure based on three years of data, and it's unclear to me how or why the department determined to use that. I do find it interesting that according to the updated definitions, charter schools look slightly better than Ohio’s Big 8 urban schools.

Steve: I think it's pretty clear why the department used the feeble federal definition -- it made them look like they had more high-quality and fewer poor quality charter schools than they actually have. So it increased their chances of receiving the $71 million. Perhaps I'm too cynical after watching this state's charter school history for nearly twenty years, but it's pretty obvious the adults at ODE tried to make our state's charter experience look like a fantasy. Again, I really hope we can hold onto the money. But I don't know if the state receives the grant if it had been totally truthful with USDOE. So I would say it was David Hansen and Richard Ross who put our children in this quandary. If the feds decide to take back the money, it is their fault, and it is they who should apologize to Ohio's kids. As for the new definition, I'm still trying to figure out this whole high quality/low quality dichotomy, since it's really an ODE concoction rather than a legislated report card definition. As for the Big 8 comparison, only about one-half of charter kids come from there anymore. Charters accepted kids from every district last year—and a significant portion of kids attend brick-and-mortar charter schools outside the Big 8 urban districts, so it's not just an e-school thing. It seems to me unfair to accept money and kids from every district, yet insist that performance comparisons be made only with the most struggling districts.

Jamie: The new definition for "high quality" is in the Ohio Revised Code, and sets the eligibility parameters for Ohio's new facilities grant for charter schools (a Value-Added grade of B or better and a Performance Index grade of C or higher, or a Value-Added grade of B or better and a Performance Index increase for the last three years). So I think it makes sense to use this definition again here. Arguably it should have been used from the beginning. I hear what you're saying about cynicism over the last twenty years. However, the purpose of CSP is to expand high-quality charter schools—not necessarily to reward states with the highest-performing sectors. In fact, Ohio's cannot becoming high-performing until we replicate and expand the best networks while simultaneously shutting down low-performers. HB 2 will help ensure that poor performers close, rather than hopping to new sponsors; the rigor of the sponsor performance review will ensure that sponsors aren't opening poorly vetted schools anymore. All of this is to say that we're on the right track. But we need start-up funds to get us the rest of the way there. Only one-half of charter kids come from the Big 8? I'd like to see more data on that. I think the gist is that charter schools serve similar percentages of kids in poverty and students of color as the Big 8 urban districts. And that's what makes it a fair comparison.

Steve: I'll give you the “high quality” definition, but poor quality was not similarly defined in statute— hence my high quality/low quality dichotomy issue. We simply don't classify charters like that here. I suppose you could assume that the automatic closure law standard would hold when determining low quality? But the department used a D or F grade standard in their letter rather than the simple F standard of the automatic closure law (which of course is on hold until at least 2017–2018, but that's another story). I'm encouraged that the department considers Ds to not be good grades. Will its “poor performing” definition have a material impact on the application? I don't know. But it's certainly concerning. And the fact that, once again, three low-performing e-schools (ECOT, OHDELA, and the Virtual Community School of Ohio) qualified for but weren't included in the poor performing definition raises the David Hansen data rigging specter. I know those schools won't get a dime (it's hoped, but we did give ECOT, which got all Fs and a D on the last report card, a Straight A Fund grant), but the USDOE wants to know the state of Ohio's entire charter school sector, not just the sliver in which they will be investing. I get that the department doesn't want to advertise that including those three e-schools means that more than 30 percent of Ohio's kids are in poor performing charter schools. But doesn't that strengthen the need to bring in more money for high-quality ones? I would think arguing that we need the money to drive down that percentage would be a pretty good argument. It doesn't help that ECOT and OHDELA are run by big political contributors. Not including them is a bad look. As a side note, I'll be curious to see whether the legislature will revisit the automatic closure law to include Ds and Fs rather than just Fs, in light of the department's apparent determination that Fs AND Ds are poor performing. 

Jamie: I believe that the new low-performing state definition (D or F in Value Added and D or F in performance index) aligns with the definition for low-performers as spelled out in HB 2 (for purposes of prohibiting schools from sponsor hopping). I think it's important to reiterate a point you made: Ohio's e-schools aren't in line for CSP funding; at least—it would be incredibly unlikely (and I don’t believe any have one past CSP grants). So I hope that the quality issues they're experiencing don't interfere with Ohio's ability to secure the funds for high-quality brick-and-mortar charters. Because they occupy such a significant percentage of our sector, as you correctly note, their scores really drag down the scores of the charter sector as a whole. And because they are so fundamentally different from brick-and-mortar charter schools (in student demographics, where they can enroll from, etc.), some might question whether they should be considered charter schools at all. But that's a conversation for another day.

My Latest Real Clear Education Piece Hits on $71 million Federal Grant

I had another piece run in Real Clear Education yesterday talking about how at risk Ohio's $71 million federal charter school grant could be in because of the adults at the Ohio Department of Education trying to hide our charter school sector's poor performance. Here's the piece from the Real Clear Education website, which is a really good, comprehensive place to find education policy news from around the country.

February 15, 2016

Education Leaders’ Lies Could Hurt Kids – in Ohio and Across America

By Stephen Dyer

_{Sen. Rand Paul, R-Kentucky, center, participates in a round table discussion on school choice with Sen. Rob Portman, R-Ohio, and Rep. Brad Wenstrup, R-Ohio, at Carpe Diem-Aiken, a tuition-free public charter school, Friday, May 16, 2014, in Cincinnati. (AP Photo/Al Behrman)}

RCEd Commentary

For several months now, many have lookedaskance at Ohio’s $71 million award last fall from the U.S. Department of Education’s Charter School Grant Program. After nearly two years of stories, reports and tales of Ohio’s struggling Charter School sector, it seemed counterintuitive for the federal government to give the state the largest chunk of a fund meant to grow high-quality charter schools. There just aren’t that many here.

Then we read the grant application the state sent into USDOE. And boy were there problems. The state claimed none of its charter schools were failing in the 2012-2013 school year, yet that year 41 percent of all charter school grades on the state report card were Fs. More than 60 percent were Ds and Fs. If Ohio should receive a grant, why not Massachusetts, which has what is widely regarded as thebest charter school sector in the country, not to mention that it’s facing a lawsuit seeking to lift the charter school cap in that state so more schools can open. That conundrum could be solved – at least partly – with some or all of the $71 million. It would seem that the charter school movement would be better served by investing in places where charters were effective, rather than Ohio where they are, on the whole, not.

So that didn’t make sense. The state claimed it had a tough automatic closure law, then made no mention that the law’s been suspended until at least the 2017-2018 school year and it hasn’tclosed but two dozen schools in 10 years. There were other problems with it, too, many of which have been outlined here and here.

The issues were troubling enough to the USDOE that they suspended payment of the $71 million and forced the state to respond to detailed questions about why certain things were said on the application that didn’t comport with fact.

Well, the Columbus Dispatch just acquired a new response sent from the Department to USDOE. In it, the Department admits it misled the feds on how many Ohio Charter Schools were failing. In the application, it claimed only six (out of more than 400 total) were failing in the 2013-2014 school year. Now, they admit that 57 were. Meanwhile, instead of 93 high-performing charters, there are 59.

However, even these data corrections come with caveats.

First, it doesn’t include online schools, which are among the worst-performing schools in the state and educate about 40,000 of the 120,000 students in Ohio charter schools. There are three online schools that would count as poor performing under the state’s definition, accounting for 17 percent of all Ohio charter school students. Not including online schools makes it seem that far fewer students are in poor performing Ohio charters than there actually are.

Second, it doesn’t include the 100 or so Dropout Recovery charter schools, which also are among the worst performing. Many don’t graduate even 10 percent of their students … in 8 years.

It is clear that David Hansen, who had to resign his post as Ohio’s head of the Office of Quality School Choice because he illegally scrubbed data to make charter school authorizers look better than they were, worked his magic again on this federal application. Hansen – whose wife is the campaign manager for the presidential campaign of Republican Ohio Gov. John Kasich, who recently placed second in the New Hampshire Primary – was in charge of putting it together. Former State Supt. Richard Ross attested with his signature to the truth of all the information contained in the application, under penalty of law.

Will these acts of malfeasance cost the kids of Ohio $71 million and a chance to put some juice into the move toward quality school options here? I hope not. This money could really help bolster the few charters we have here that are fulfilling the idea’s promise and give momentum for the state’s newfound push for quality. And as I’ve said before in this space, if Ohio can get its act together on charter school quality, it would have tremendous national implications because it would prove that charters could work anywhere. But if the federal money was granted based on lies, should it still be granted now that the truth is slowly seeping from thebeleaguered department—that Ohio doesn’t have lots of high-performing charters in which to invest the federal money meant to expand their footprint?

Perhaps it should be sent to other states that didn’t mislead on their grant applications, as Ohio did.

If the money is taken off the table by the feds, there is no one to blame but the adults at the Ohio Department of Education who were so blinded by ideology they ignored the facts everyone else saw clear as day. This is a lesson for our country’s ideologues on all sides of the charter debate. We shouldn’t let our biases color the truth. If they do, kids in states that don’t cheat will suffer. And kids in states that do cheat will also suffer because their state’s inferior charter sector will expand at the expense of another state’s higher performing sector.

Either way this turns out, those adults who misled under oath to win federal money should be held to account for what they did to Ohio’s kids. For if they are able to escape consequence, what’s to stop future applicants from giving similarly evasive answers to USDOE, ultimately driving down the program’s effectiveness expanding high-quality charter schools? We must not forget that this money is our tax money. We should insist on absolute transparency not only in how it’s spent, but how it’s awarded.

We teach kids to not cheat. Our adults, it seems, need a refresher course.

Stephen Dyer – a lawyer – is the Education Policy Fellow at Innovation Ohio – a progressive Columbus, Ohio think tank. He served four years in the Ohio House, focusing on education policy and chairing an education subcommittee dealing with finance and reform. His work reforming the state’s education funding, human capital and accountability systems earned Ohio the Frank Newman Award for State Innovation from the Education Commission of the States in 2010, given annually to the country’s most “bold, courageous (and) bipartisan” education reform effort. Prior to that, he reported for nearly a decade on education and other issues with the Akron Beacon Journal