Accountability for Thee....

Another round of "accountability for teachers but not for administrators"?

According to the DCPS IMPACT Guidebook, the actual growth is a student’s scaled score at the end of a given year minus his or her scaled score at the end of the prior year. If a fifth-grader received a scaled score of 535 in math and a score of 448 on the fourth-grade test the previous year, his actual gain would be calculated as 87 points.

Subtracting one score from another only makes sense if the two scores are on the same scale. We wouldn’t, for example, subtract 448 apples from 535 oranges and expect an interpretable result. But that’s exactly what the DC value-added approach is doing: Subtracting values from scales that aren’t comparable.

By assuming that the difference between a student’s score in one year and his or her score in the following year is a definite (and precise) quantity, the DCPS value-added scheme assumes that the scaled scores are measured on an interval-level scale, in which the difference between a score of 498 and a score of 499 represents the same difference in performance as the difference between 499 and 500.

But this simply cannot be. The difference between 498 and 499 is a tiny difference among very high achievers in the fourth grade. But the difference between 499 and 500 is the difference between the highest performing fourth-grader and the lowest performing fifth-grader; and there are many fourth-graders who outperform low-scoring fifth-graders.

And heaven help the poor teacher who is teaching a class filled with students who’ve been retained in grade.

A fifth-grade student who got every question wrong on the reading test at the end of fourth grade and every question wrong at the end of fifth grade would show an actual gain of 500–400=100 points.

A fifth-grader repeating fifth grade who had a scaled score of 510 the first time through, and a scaled score of 530 during his or her second year in fifth grade, would show an actual gain of just 20 points. DC’s value-added methods may, of course, simply exclude students who are retained in grade from the calculations, but that sends an unpleasant message about whose scores count when teachers are evaluated.

I'll leave open the possibility that the D.C. Schools' calculation of value-added scores was not as incompetent as it seems from the data they've released so far, but....