Thursday, October 01, 2009

Premature Triumphalism

The Washington Post comments on a study about the performance of charter schools in New York City,
Researchers were careful not to draw conclusions, but they highlighted a correlation to practices such as a longer school day, performance pay for teachers, more time spent on English and effective discipline policies.
That's perfectly reasonable - and hardly surprising. More specifically, the study states,
We are cautious about all of the associations with achievement that we describe above. First, these associations may change as more data are added or more New York City charter schools open. Second, the associations can be difficult to interpret because some policies are routinely found together in packages: an example is the package of a long school year and long school day. It is difficult to disentangle the role played by each part of a package. Third, it is essential to remember that none of the associations we have described is a causal effect. We are not asserting that if a school adopts a certain policy, its achievement effects will rise. One must have causal effects to make such assertions. We cannot tell whether the policies themselves make the difference or whether the policies are merely correlated with factors that do make the difference (great leadership and so on). We strongly discourage readers from treating the associations as though they were causal effects--for instance, by changing a policy based on the estimates shown.

With all these caveats, the positive associations are with a long school year (this is especially strong), the number of minutes spent on English per day, a small rewards/small punishments disciplinary strategy, teacher pay based on performance or duties, and a mission statement that emphasizes academic performance.
None of that's a surprise and, given funding, many of those ideas could be implemented in public schools. So, having acknowledged that the study's authors warn against hasty assumptions or conclusions, how does Fred Hiatt's editorial board respond to these preliminary, and largely intuitive, findings?
Poor children learn. Teachers unions are not pleased.
Um... okay. Well, perhaps that's just a catchy little tagline meant to attract more readers. What else do you have, Fred?
A rigorous new study of charter schools in New York City demolishes the argument that charter schools outperform traditional public schools only because they get the "best students."
You know, the study is worthy of discussion. It introduces an element to the comparison of charter schools and public schools that has been missing from prior studies - an attempt to compare students who have been lotteried in versus those who lost the lotteries and remained in public schools. Not one of the correlations highlighted by the study relate to teaching technique, and not one of the correlations couldn't be implemented in public schools... given increased funding and, in some circumstances (e.g., with increasing the school day or year, or implementing merit pay), following the next union contract. The hardest factor is discipline - charter schools can do things to discipline students and maintain order that public schools cannot - but steps can be taken to improve order in schools and classrooms. It's hardly a surprise that students who are in a distracting, poorly disciplined school or classroom environment fall behind their peers who enjoy calm, orderly learning environments.

What this, the Post's "rigorous new study", highlights is how little research has been done into charter schools, what they contribute, how they compare to public schools, and what techniques can be translated from one set of schools to another. If this is to be taken as proof that underperforming schools should be closed in favor of more charter schools, the Post should be making a perfectly reasonable associated demand: that underperforming charter schools have their charters pulled. I have to tell you, the top performing charter schools posted impressive gains for their students. The Post's case would be much better if they were all performing at that level. We must also remember that we're talking about a school system that's pretty bad; when closing "about 86 percent of the 'Scarsdale-Harlem achievement gap' in math and 66 percent of the achievement gap in English" is trumpeted a great success, you know that the baseline is way below standard.

The study itself is worth reading, and its actual findings are definitely worthy of consideration and additional study. I would go so far as to say, with due respect to the study's caution against "changing a policy based on the estimates shown", underperforming school districts should be contemplating how to implement the factors identified as improving student performance (and newspapers like the Post, if they're going to offer more than words in support of education reform, should back their receiving the necessary funding to do so), and perhaps even implementing them as common sense reforms while we wait for further data. It doesn't seem to me that a whole lot of bad can come out of, for example, a longer school day or school year (assuming infrastructure, compensation and staffing issues are adequately addressed).


  1. Thank you for bringing up the issue of how charters cherry pick students. As I'm sure I've said before, charters (esp. in Detroit) routinely take special ed kids until after Count Day, and then send them back to us in droves. Now, if they can take students who are LD and students who are EI, CI and EIEIO and make real progress, well, then that's certainly something.

    What I'd like to know is how many teachers are there b/c they want to be. I only have a very small worldview here, but let me tell you that in years of education classes, I have met NO teachers who want to work in a charter school. They will end up there, but I have never met one that wants to work for one. Why would you? There's no job security, you can get fired for farting in the wrong way, it's about 1/2 the pay, no pension, long hours...uh, it's like legal aid law for God's sake :) Again, I know there are teachers who want to work in charters, but I've never met one.

    The longer year thing, while interesting, may not work out in my district. Come to our schools at the end of the year and you will see that many kids stop coming in early June. I don't know how you overcome that mindset. But I can say that if you lengthen our school year, a lot of kids would just stop coming, thus negating any real progress. Also, I am a huge fan of intensive summer school, like my district had last year. That way, kids who don't need to be there or kids who need to work in the summers are not forced to be there.

    We have to remember that at the heart of it all, a charter school is FOR profit and exists to make money for someone. That's the part that I really can't live with.

    (For some admittedly biased charter school bitching, check out the teacher chatboards on Some of the stories--if true--make me want to cling to my public school job for dear life)

  2. I should have mentioned that the study focuses principally on K-8, due to the low number of charter high schools. In terms of students who stop attending school toward the end of the year, to the extent that's a problem in K-8 classrooms it's reasonably classified as a parental failure, not a school failure. However, I am not sure that the effect is relevant to this particular study, as this study attempts to compare kids whose parents all wanted them to go to charter schools, at least to the extent of entering them in at least one lottery.

    The attempt here was to determine whether a body of students roughly equivalent to the students admitted into charter schools didn't fare as well academically as their charter school peers, not whether charter school students can "outperform" students whose parents don't even care if their eighth graders finish out the school year. If charter schools couldn't do that, I doubt even the Post would be pushing them. (I suspect the Post's emphasis on charter schools has something to do with its Kaplan subsidiary.)

    It would be interesting to hear how NYC's charter high schools compete against its magnet high schools, which admit based upon academic testing. I suspect that between those magnet schools and the higher cost of running a high school, there's less room for charters to either turn a profit or claim superior outcomes.