Share it

Thursday, July 31, 2014

More Horizon/Charter Test Questions

After the allegations at Horizon Science Academy's Dayton campus were raised a couple weeks ago, I decided to look at Horizon's other schools to see if there was any evidence of test manipulation, similar to what was alleged to have gone on at Horizon's Dayton campus.

The way I looked at it was to compare the school's Performance Index (PI) score (it's state test results) with its ACT score (it's college entrance scores, which are handled through the College Board). The idea was that the school should score about the same on each. And, in fact, that's what happens in Ohio school districts. The correlation coefficient between a district's PI and ACT results is .79, which by most measures qualifies as a "strong" positive correlation, meaning the higher the PI, the higher the ACT, and vice versa. That .79 is about the same strength of correlation as exists between education level and income, just to give you a sense of scale and strength.

So in Ohio school districts, the state tests and ACT track pretty closely with one another.

Here's the pertinent info from the report I authored for Innovation Ohio:

"According to the latest State Report Card data, Horizon Science Academy Columbus scored a 98.6 on the state’s Performance Index (PI), placing it just below the state average score of 98.9 for all public school districts. The score was the highest of the four Horizon academies we reviewed. However, the average ACT score at Horizon Columbus was just 17, placing the students who took the test near the bottom in the 28th percentile."

Adding complication to the Horizon Columbus issue is that they are a National Blue Ribbon Award winning school. The Blue Ribbon school program was started in 1982 and recognizes the top performing schools in the country. The issue is that the award is based on test scores. If Horizon Science Academy Columbus fudged their scores, as is being alleged in Dayton and which their comparative test data suggest may be happening in Columbus, what does that do to the award?

And to go one step further, in Ohio's Charter Schools, those two data points are decidedly less strongly correlated, with a coefficient of .41. (The closer to 1 the correlation, the stronger the relationship) There remains a positive correlation, it's just much weaker in Ohio's Charter Schools. To be fair, there are only 18 of 281 Charter Schools on the Ohio report card that report having ACT scores, while all 612 school districts do. So with fewer datapoints, the Charter School data is more susceptible to being influenced by extreme examples.

However, Charters are all over the place on the two scores. Take the Arts and College Preparatory Academy in Franklin County. Their PI of 108.3 rivals the best Ohio school districts. Yet their 19 average ACT score (41st percentile nationally) is about 5 points lower than the 23.9 average ACT for the 10 school districts whose PI scores are between 108 and 109. The lowest district ACT score was 21 (55th percentile), the highest average ACT was 26 (84th percentile).

There were 543 school districts that scored better than Arts and College Prep on the average ACT, yet only 21 school districts scored higher on the PI. However, remove Arts and College Prep, and the correlation in Ohio's Charter Schools jumps to .46. Still not as strong as OHio's traditional public schools, but still stronger. And you can see what an influence extremes can have when you have fewer data points.

However, now that Auditor of State David Yost will be looking into the test disparities in the Horizon Columbus school as a result of our report, will he expand that to the other Charter Schools whose disparities are so striking?

As I made clear in our report, test disparity does not prove any funny business. But it sure merits examination by state regulators. And that should include traditional public schools too, by the way. However, the overall trend in traditional schools' scores suggests less funny business than the overall trend in Ohio's Charter Schools.

So stay tuned.

Thursday, July 17, 2014

Beyond the Horizon

There's been significant attention paid to the really outrageous allegations from the Gulen schools. Between covered up sexual misconduct and test score fixing, there's plenty of outrage to go around. And now that we know the presumptive future Ohio House Speaker has gone on junkets with those who run the Horizon and Noble academies, there are real questions about this legislature's willingness to do anything about it.

But I want us to look beyond these outrageous actions. Because this stuff could never go on at a traditional public school. Why? Transparency. And in Ohio Charter School Land, there simply isn't any. Don't get me wrong, there are a scant few charters that are open and transparent, but in Ohio they don't HAVE to be. Sponsors are responsible for holding Charters to account. But if they don't want to, they really don't have to. That needs to change. Now. How many stories of middle schoolers having sex and schools covering it up to parents do we have to hear before this legislature does anything? How many clear calls of test manipulation? Or misappropriation of state funds?

When will this state wake up? It's public money. Charters are supposed to be public schools. Let's make sure the public can find out what the heck is going on at these things. It shouldn't take whistleblowers. It should take a simple public inspection.  Period.

Tuesday, July 8, 2014

Beware Xenophobia with Embattled Gulen Charters

On Sunday, the Akron Beacon Journal reported on some disturbing trends with several Charter Schools run by an organization closely tied with the the Gulen movement out of Turkey -- a movement that is as complicated as one would expect from the Middle East. There are those who say that the Gulen movement promotes an interfaith collaboration that can help temper religious strife in a region riven with it. In fact, two professors of religion (one at Temple University and one at the the Lutheran Theological Seminary at Philadelphia) said this of the movement:
Our own research, based on years of familiarity with the writings of G├╝len, and associations with Turkish businessmen, scientists, and civic leaders, suggests ... these schools have consistently promoted good learning and citizenship, and the Hizmet movement is to date an evidently admirable civil society organization to build bridges between religious communities and to provide direct service on behalf of the common good.
Yet the movement was booted from its native Turkey by the current government for being accused of trying to set up an Islamist state in Turkey. Or were they booted because they were too politically powerful for the current ruling class, not unlike how other states have exiled or imprisoned dissidents throughout the millenia?

Critics have charged the movement with being more like a cult, adhering blindly to Gulen's message.

So is the Gulen Movement a moderate form of Islam that can counterbalance the more strident voices in the religion? Or is it a Heaven's Gate type of cult?

I don't know. But the answer is much more complicated than many folks who are now, emboldened by the Beacon story, going after the Gulen schools in Ohio for, among other things, using lots of Turkish teachers in the classroom. The Vindicator, in an I-told-you-so editorial, said that they had been warning for years that "there is no way for the state of Ohio to prevent non-Christian organizations from seeking to establish charters." Like that was the biggest problem the Beacon Journal reported, never mind the numerous non-religious irregularities it uncovered.

Some have even said that the state should now make it illegal to have non-citizens on Charter School boards or even teach in classrooms.

I have to admit that I've been wary of this story because I fear it will turn into a xenophobic attack on a not-so-well-understood group.

And I think foreigners can add incredible richness to our schools.

It would be incredibly difficult for the Campus International School in Cleveland to find the four Mandarin teachers they currently have if they couldn't recruit in China. I just spent two weeks in Chinese schools that couldn't wait to bring over American teachers to teach in their schools and send their teachers to America in exchange. What better way to foster a cooperative, peaceful world than the free exchange of intellectual capital?

If the Sorbonne wanted to set up an experimental French language school in Columbus, would we want to prevent that? Or how about bringing over an education expert from Oxford or Cambridge to help run a new, innovative school in Dayton? Would we really not want them educating our kids because they spoke a weird form of English? Of course not.

My senior year of High School, Gareth Morrell, who was the chorus master at the Cleveland Orchestra at the time and a British citizen, came in and taught some of our choral classes. Would we want to deprive children of that experience?

That's why we must practice restraint and focus on the numerous irregularities at the Gulen schools that the Beacon pointed out that have little to do with the heritage of the founders and more to do with their ethical makeup. Things such as test alterations (which caused so much consternation for lawmakers about Columbus City Schools), or that the teachers they are bringing over aren't qualified to teach, or that the Speaker of the House in-waiting Cliff Rosenberger was wined and dined by this group, especially since current Speaker Bill Batchelder has been notoriously in the pocket of American citizen and far greater taxpayer Charter School scofflaw David Brennan (who's made more than $1 billion running poorly performing Charter Schools since 1998, dwarfing the amount spent on Gulen Schools by magnitudes of 10), or that the Gulen Schools' spokesman won't answer questions.

Those are outrages. The fact that they are being committed by Turkish, or Muslim men is less of a concern for me. What concerns me much more is that they are being committed at all.

In short, if you are as concerned about these irregularities at Gulen Schools as I am, then please do your best to avoid tapping into the public's inherent xenophobia and distrust of all things Islam and foreign. There's plenty to go after these schools for without bringing their religion or nationality into it.

For I fear if we do give in to our xenophobic inclinations, we will miss the real scandal -- that taxpayer money is once again being spent on Charter Schools that, regardless of religious or national affiliation are doing a really poor job educating Ohio's children and instead enriching people who's primary interest in education is making money, not better educated citizens. That, my friends, is a scandal far more egregious than having a few foreign nationals sitting on some Charter School boards.

Monday, July 7, 2014

Beleaguered ECOT gets another Innovation Grant

In the latest round of Straight A Fund recipients, ECOT came out another winner, it appears. They weren't the primary recipient of the $625,000, the way it was during the first round when it got $3 million. Instead, they partnered with Finneytown Local in Hamilton County.

Finneytown, according to payment reports from the Ohio Department of Education, lost a bit more than $18,000 to ECOT last school year. ECOT cleared $99.3 million from school districts last year -- more than $6,800 per pupil (far more per pupil state money than the average traditional public school receives).

So to keep the running total, since last year's budget, ECOT was awarded the state's largest Charter School funding bump from Gov. John Kasich's new education funding plan, as well as significant casino revenue and two Straight A grants. That means that ECOT has received an 11% per pupil funding bump since Kasich's education funding plan was announced.

Since ECOT's report card ratings are extremely mediocre and its graduation rate is an abysmal 35%, something other than academic excellence must be at work.

ECOT founder William Lager has made $368,000 in political contributions since April of last year. So that begs the question: Has ECOT really been doing a great job, or has its founder done a better one investing in politicians?

Charters Least Efficient Straight A Recipients

These posts are starting to get old for me. I really hope that our state leaders start doing something about this stuff soon.

Anyway, in the latest Straight A Fund round (Ohio's version of Race to the Top, sort of), there was a $2.075 million award given to a group of Charter Schools in order to "implement a data and assessment system that will help improve student achievement." Now I'm all for that, I guess, but here's the kicker: Of the 37 grantees, no award will realize a smaller amount of cost savings relative to the taxpayer investment than this Charter School proposal.

Each award has a cost and anticipated cost savings attached to it. The Charter Schools' proposal was $2.075 million, with an anticipated savings of about $117,000 -- or 5.7% of the cost. The average savings of all the 37 awards was more than 250%, or nearly 45 times greater than the Charter Schools' proposal.

Here are some examples: The Central Ohio ESC will implement a system that will better match students with adults as they prepare for college and career. The cost savings will be $43 million on the $8.8 million investment. Or an initiative done through Columbus Public Schools in cooperation with Johns Hopkins that will see a 1313% return on the $677,000 investment.

For full disclosure, I was a Straight A reviewer for the first round of applications. And my major criticism of the process is that I couldn't give passing marks to an applicant for fiscal sustainability if the applicant was going to use the money to create additional revenue streams. The only thing I could consider was how much money would be saved. So the fund is essentially incentivizing schools to make permanent cuts in their programming.

So I'm not overly enamored with judging these recipients based on their rate of return. However, am I the only one who finds it curious that the "bloated" public sector recipients have such a better return on investment than their "efficient" private sector counterparts?

If you've been reading this space with any regularity, you shouldn't. But it still makes me wonder why the heck our state leaders are so blind to the obvious -- that Ohio's Charter School system is simply not working, not for our taxpayers and certainly not for our kids.

It's time to seriously reform the reform.

Tuesday, June 17, 2014

Happy Birthday Standardized Tests!!!! Um...yeah.

My dad pointed out something to me on Facebook today: June 17 marks the 113th birthday of the American standardized test. I have to say that our country's education system has never really been the same since.

While China has had standardized tests for centuries, introducing them in America has been fraught with issues, which are well documented. The most important to me, frankly, is whether they actually predict future life success, or merely tell us who is set up to succeed in the first place.

For example, using an Excel regression analysis, I can predict 3 out of 4 Ohio school district standardized test performance ratings on the state's Performance Index Score. All I need to know is a district's percentage of children living in poverty, or their median income.

And despite the fact that American kids have never done well on international tests, the last 40-50 years has been dominated by the American economy (still #1 on GDP), we won the Cold War, and America has dominated the production of intellectual property.

So do these tests scores really tell us that we suck, or are in crisis? Do they tell us anything that's useful or predictive?

My son, who is 9, already has test anxiety because of the number and importance of these standardized tests. I'm seriously considering opting him out of future tests until high school.

However, not all testing is necessarily bad. Used to assess kids so teachers can better direct learning toward individual student needs is a great use, for example. Using them to assess teachers and classrooms? Not so obviously great.

However, I have great sympathy for the many parents I have met whose children were ignored until standardized testing -- and the accountability that came with them -- forced schools to pay attention and teach them. That has always been the standardized testing result that has kept me from advocating their immediate burning, though I certainly have a book of matches at the ready.

Then there's the promising work of Robert J. Sternberg, who was at Tufts University  for awhile (my alma mater -- shameless plug. By the way, we took him from Yale. How's that taste, New Haven?). Sternberg has developed tests that assess the creative, analytical and practical skills of students. One study from his work in the 1990s found that the students who did well on the analytical portion of the tests had the background of those who perform well on our standardized tests today -- more wealthy and white. However, on the creative and practical portions, the results showed a great mix of diverse socio-economic and cultural backgrounds. Other studies have verified those results.

Imagine if our high-stakes regime measured the other two categories? Maybe, then, traditionally low-performing Cleveland wouldn't qualify for Vouchers and Charters because their students do so well on creative and practical skills, while perhaps traditionally high-performing Hudson would qualify because their students don't?

Imagine that paradigm shift, eh?

As our testing regime becomes more high-stakes, and more importance is placed on succeeding on these things, perhaps it's time to develop tests that measure the whole student, not just their ability to analyze things and memorize. There's more to life than that, and there's more to schooling than that.

Until that happens, though, I fear all our testing regime will do is narrow curriculum, minimize the importance of the kind of thinking that can lead to true innovation and success, and force us into becoming a nation of rote learners who can't come up with ideas, but sure can tell you everything you need to know about someone else's.

Unless, that is, parents, administrators, teachers and others stand up and say, "No more!"

Wednesday, June 11, 2014

CA Court Whiffs on Key Question

A California judge ruled yesterday that so-called "first in, last out" teacher laws violate students' equal opportunity to have Great Teachers. And while many commentators have found this to be a much-needed repudiation of the "old guard" teacher labor paradigm, dominated by unions, I can say that as the son of a former teacher who actually would have been laid off under a "first in, last out" system at a newspaper, the ruling is amazing for its exceptional blind spot: 

We have no idea what a Great Teacher for these kids would look like. 

While there are scant few obviously Great and obviously Bad teachers, the overwhelming number are neither. Yet we act like there's a Platonic Form of a Great Teacher out there that will be Great for Every Kid in Every School, and the only thing standing between our kids and that Platonic ideal is a Teacher's Union.

But is that the case?

I mean, we all know that Aristotle was a Great Teacher for Alexander, but would Aristotle have been a Great Teacher in front of 60 kids, 7 times a day in an 8th Grade building on the South Side of Chicago, or in Boone County, West Virginia, or in Beverly Hills? 

I don't know. 

And this is the huge logical hole that remains at the core of all this "a Great Teacher for every kid" argument. We talk about Great Teachers like we know what one actually is, or some magic metric can tell us, or that we actually agree on the Platonic Form that teacher would take. It's a given that Great Teachers should be teaching our most at-risk kids just as frequently as our most privileged. However, the assumption many make is that Great Teachers for privileged students would be just as Great teaching at-risk kids. The ideal, Platonic teacher for one kid is the ideal for every kid, so goes the assumption.

Yet nothing could be less certain. Ask parents who request teachers for their kids based on sterling recommendations from well-respected friends and educators, then realize their kids aren't a match a month into what ends up being a very ordinary school year. 

Subjectivity in teacher quality is the wild card in determining teacher effectiveness. And sometimes, kids aren't aware of how Great their teachers are until far down their life road.

For example, I had two high school teachers that I really didn't like at the time. They demanded that I practice, work, study and do all the things an immature teenager doesn't want to do. I mean, I really hated going to Mr. Appling's music and Mrs. Pryce's French classes. Because I knew I hadn't done enough preparation to their standard, but I did enough to get by, so why wasn't that good enough? They're so mean!

Well, they knew I could do better. They expected more from me, even if I didn't expect it from myself.

Now, if you let me do a teacher evaluation of these two phenomenal teachers at the time, I would have excoriated them. I would have hammered them for their tough teaching styles and "unreasonable" expectations. And I wouldn't have been alone. Everyone I knew would have given them the same marks.

Today, I recognize that they were Great Teachers because they showed me how hard I had to work. But if they were being judged on student evaluations, as many want us to do, they wouldn't have lasted a year.

Likewise, I have witnessed teachers who spend their entire years drilling their students for state achievement tests, with very little other curriculum. Their kids will score off the charts and show great growth. They actually brag about how highly their students score. But is that a Great Teacher? Or is it just Great test preparation? Under the proposed tying teacher evaluation to test scores regime, those teachers will be considered Great Teachers. 

However, I beg to differ.

The more I think of what a Great Teacher is, the more I think of the famous quote from U.S. Supreme Court Justice Potter Stewart when he described pornography: "I know it when I see it."

And what's even more complicated about evaluating teachers is one student's Aristotle is another's Elizabeth Halsey (look at IMDb for that reference). I can't tell you how often I would defend teachers I loved to classmates who hated them. And vice versa.

Adding more complication is that some Aristotles are Great Teachers in one environment and not so Great in others.

Some of the best teachers I ever had would have been far less successful in larger classes, for example, because they thrived on interactive discussions. I've had fantastic teachers in 300 student lectures, but they would have been not nearly as successful in smaller settings because they didn't like interaction at all.

Which brings me back to the California case. I have deep sympathy for kids who are in schools where they feel they're being denied access to the Great Teacher they deserve because of some outdated, union protection. But there is no guarantee that these "Great Teachers" will be Great teaching these students. They might even be worse. 

We just don't know.

The "first in, last out" paradigm is actually better than any we've come up with yet because it's about as objective as can be. It doesn't play favorites. It doesn't require a friendship with a principal. It doesn't force teachers to do combat with each other about who gets which kids so their test scores can be the highest on their evaluations.

Is it perfect? Far from it. It produces inequities. It lets go of good, great teachers even. But absent a truly substantive, thorough evaluation system, it's the best, most objective system we've got. That's why I didn't complain much about potentially being laid off at the newspaper I worked at because I was more junior than others. I didn't have to worry about whether I had a certain number of bylines, or whether my work led to a certain numbers of legislative changes. Or whether I won awards (which I did several times). Or whether I dangled too many participles. I just did my job the best I could, and if the economy wouldn't support my employment at the paper, so be it. 

That fact spurred me to go to law school so I would have something to fall back on if the layoffs did happen. Luckily, I found an exciting new career before the layoffs hit my level of seniority. 

Likewise, when you get into substantive evaluation of teaching excellence, it is so much more complicated than how a teacher's kids score on tests, or whether the kids like them, or whether the parents feel like the teachers are their friends. 

It's really, really complicated. Really.

The best system I can imagine is one that does take test scores, teacher and parent evaluations into some minor account, but is overwhelmingly driven by close evaluation by principals and lead teachers. They'll be best able to identify teachers who are best matched to each environment. 

It is through deep, meaningful evaluation that teachers will be placed most effectively in the classroom, or let go. However, without proper resources to accomplish these meaningful evaluations, principals will be stretched to their limits, evaluating teachers quickly because they simply don't have time to evaluate everyone the way they need to be. 

And that will lead to inequities as well, equally as egregious as the ones mentioned in the California case.

(Need I mention that the greatest inequity is how schools are funded? But that's another story for another day.)

The silver lining I see in the California case is this: It could force policymakers and educators to figure out a more thorough, meaningful evaluation system that provides a better, more complete picture of what a Great Teacher looks like for which kids, and where they're best placed. Yet without financial backing, I fear these systems won't be implemented properly, and we will have continued inequities in the system, with children being denied access to the Great Teacher they need and deserve.

Because right now, if you asked me, "What makes a Great Teacher?" I would have to give the Potter Stewart answer, with a twist:

I know it when I see it. Even if I don't realize it until 10 years later.