A Feel Good Story About Teachers....

... That's isn't properly supported by the data. Yes, it's nice to think that a kindergarten teacher can have a profound effect on the future incomes and success of her students, and yes there probably is some truth to it - it makes sense that your future education will build upon the foundation laid in kindergarten. It also makes sense that a more positive kindergarten experience, even if it's simply at an emotional level, will make it more likely that a child will come into future years of schooling with the expectation of further positive experiences. But...

The crucial problem the study had to solve was the old causation-correlation problem. Are children who do well on kindergarten tests destined to do better in life, based on who they are? Or are their teacher and classmates changing them?

The Tennessee experiment, known as Project Star, offered a chance to answer these questions because it randomly assigned students to a kindergarten class. As a result, the classes had fairly similar socioeconomic mixes of students and could be expected to perform similarly on the tests given at the end of kindergarten.

Yet they didn’t. Some classes did far better than others. The differences were too big to be explained by randomness. (Similarly, when the researchers looked at entering and exiting test scores in first, second and third grades, they found that some classes made much more progress than others.)

Class size — which was the impetus of Project Star — evidently played some role. Classes with 13 to 17 students did better than classes with 22 to 25. Peers also seem to matter. In classes with a somewhat higher average socioeconomic status, all the students tended to do a little better.

But neither of these factors came close to explaining the variation in class performance. So another cause seemed to be the explanation: teachers.

There are two issues here. The first is that the mere fact that the statistical significance of a finding is supposedly "too big to be explained by randomness" doesn't mean that it isn't, in fact, the result of chance. A very interesting New Yorker article recently discussed a number of studies in which statistically significant results, sometimes well beyond what researchers believed could be explained by chance, were followed in subsequent studies by a regression to the mean. That is, the initial findings were outliers or were otherwise flawed. The article doesn't mention this, but sometimes data sets are analyzed by computer for hundreds or thousands of factors - and when you do that, odds are you're going to find some outliers at the p < .01 or even p < ..001 level.

Also, with due respect to Sherlock Holmes, ruling out the obvious does not necessarily mean that you are correct that only one explanation remains, or that the one explanation you identify is the solution to your puzzle. If you come up with a hypothesis that lies outside of your data, you should attempt to replicate your experiment while collecting a more complete set of data.

Even if the answer does lie with the quality of the kindergarten teacher, the questions are raised, what qualities made the difference and were they consistent between teachers? If some teachers were introducing basic math and reading concepts, while others were focusing on play and socialization, that could explain why one group performed better than the other, but that's really a question of priority as opposed to teacher quality.

When it comes to schools we're better off erring on the side of quality, top to bottom, so to the extent that "great kindergarten teachers cause significant increases in lifetime earnings" means that as a society we'll emphasize finding and retaining great kindergarten teachers, I'm all for it. But, perhaps thanks to my kindergarten teacher (although when I started school in the U.K. I didn't attend a kindergarten, as such) I don't accept "We can't figure out what else caused the difference so it must be the teachers" with sound science.