I had the opportunity to study under Lipsey when he was one of the instructors at the IES CRT Summer Institute. That he of all people had to deal with this is really depressing.
Do not question the narrative that all educational spending and programs are double-plus good! We must think of the children!
We believe that most people will agree that society has an obligation to prevent or ameliorate the harmful effects of poverty on children. U.S. policy makers have shown little interest in prevention, but have been broadly supportive of preschool as an approach to amelioration. Head Start has this same mission, but a movement emerged in the 1980s to provide pre-k under the supervision of the public schools, with further expansion beginning in the early 2000s. However, the only research evidence available to guide states on whether such programs would be effective came from small, boutique researcher-driven programs implemented 40-50 years previously. Although the state programs were not like these earlier experimental projects, the data on their long-term effectiveness was proclaimed loudly and frequently to justify state funding.
In 2008 we worked closely with the Tennessee Department of Education to craft a strong experimental design that would assess the effectiveness of the TN Voluntary Pre-K program (TNVPK). Other than the Head Start Impact study, this would become the only randomized control study of a scaled-up public pre-k program.
Our initial results supported the immediate effectiveness of pre-k; children in the program performed better at the end of pre-k than control children, most of whom had stayed home. The press, the public, and our colleagues relished these findings. But ours was a longitudinal study and the third grade results told a different story. Not only was there fade out, but the pre-k children scored below the controls on the state achievement tests. Moreover, they had more disciplinary offenses and none of the positive effects on retention and special education that were anticipated.
Those findings were not welcome. So much so that it has been difficult to get the results published. Our first attempt was reviewed by pre-k advocates who had disparaged our findings when they first came out in a working paper – we know that because their reviews repeated word-for-word criticisms made in their prior blogs and commentary. We are grateful for an open-minded editor who allowed our recent paper summarizing the results of this study to be published (after, we should note, a very thorough peer review and 17 single-spaced pages of responses to questions raised by reviewers). We are also appreciative of the objective assessment and attention to detail represented in the Straight Talk review.
It is, of course, understandable that people are skeptical of results that do not confirm the prevailing wisdom, but the vitriol with which our work has been greeted is beyond mere scientific concern. Social science research can only be helpful to policy makers if it presents findings openly and objectively, even when unwelcome.
We share with our colleagues a commitment to the goal of providing a better life for poor children. Blind commitment to one avenue for attaining that goal, however, is unnecessarily limiting. If pre-k is not working as hoped and intended, we need to roll up our sleeves and figure out what will work, with solid research to guide that effort.
