Premature Triumphalism

The Washington Post comments on a study about the performance of charter schools in New York City,

Researchers were careful not to draw conclusions, but they highlighted a correlation to practices such as a longer school day, performance pay for teachers, more time spent on English and effective discipline policies.

That's perfectly reasonable - and hardly surprising. More specifically, the study states,

We are cautious about all of the associations with achievement that we describe above. First, these associations may change as more data are added or more New York City charter schools open. Second, the associations can be difficult to interpret because some policies are routinely found together in packages: an example is the package of a long school year and long school day. It is difficult to disentangle the role played by each part of a package. Third, it is essential to remember that none of the associations we have described is a causal effect. We are not asserting that if a school adopts a certain policy, its achievement effects will rise. One must have causal effects to make such assertions. We cannot tell whether the policies themselves make the difference or whether the policies are merely correlated with factors that do make the difference (great leadership and so on). We strongly discourage readers from treating the associations as though they were causal effects--for instance, by changing a policy based on the estimates shown.

With all these caveats, the positive associations are with a long school year (this is especially strong), the number of minutes spent on English per day, a small rewards/small punishments disciplinary strategy, teacher pay based on performance or duties, and a mission statement that emphasizes academic performance.

None of that's a surprise and, given funding, many of those ideas could be implemented in public schools. So, having acknowledged that the study's authors warn against hasty assumptions or conclusions, how does Fred Hiatt's editorial board respond to these preliminary, and largely intuitive, findings?

Poor children learn. Teachers unions are not pleased.

Um... okay. Well, perhaps that's just a catchy little tagline meant to attract more readers. What else do you have, Fred?

A rigorous new study of charter schools in New York City demolishes the argument that charter schools outperform traditional public schools only because they get the "best students."

You know, the study is worthy of discussion. It introduces an element to the comparison of charter schools and public schools that has been missing from prior studies - an attempt to compare students who have been lotteried in versus those who lost the lotteries and remained in public schools. Not one of the correlations highlighted by the study relate to teaching technique, and not one of the correlations couldn't be implemented in public schools... given increased funding and, in some circumstances (e.g., with increasing the school day or year, or implementing merit pay), following the next union contract. The hardest factor is discipline - charter schools can do things to discipline students and maintain order that public schools cannot - but steps can be taken to improve order in schools and classrooms. It's hardly a surprise that students who are in a distracting, poorly disciplined school or classroom environment fall behind their peers who enjoy calm, orderly learning environments.

What this, the Post's "rigorous new study", highlights is how little research has been done into charter schools, what they contribute, how they compare to public schools, and what techniques can be translated from one set of schools to another. If this is to be taken as proof that underperforming schools should be closed in favor of more charter schools, the Post should be making a perfectly reasonable associated demand: that underperforming charter schools have their charters pulled. I have to tell you, the top performing charter schools posted impressive gains for their students. The Post's case would be much better if they were all performing at that level. We must also remember that we're talking about a school system that's pretty bad; when closing "about 86 percent of the 'Scarsdale-Harlem achievement gap' in math and 66 percent of the achievement gap in English" is trumpeted a great success, you know that the baseline is way below standard.

The study itself is worth reading, and its actual findings are definitely worthy of consideration and additional study. I would go so far as to say, with due respect to the study's caution against "changing a policy based on the estimates shown", underperforming school districts should be contemplating how to implement the factors identified as improving student performance (and newspapers like the Post, if they're going to offer more than words in support of education reform, should back their receiving the necessary funding to do so), and perhaps even implementing them as common sense reforms while we wait for further data. It doesn't seem to me that a whole lot of bad can come out of, for example, a longer school day or school year (assuming infrastructure, compensation and staffing issues are adequately addressed).