The impact of ground-up, system re-design post-Katrina

In the past decades, politicians worldwide have embarked on reforms to increase market and government accountability in state-funded education sectors. The idea behind has been to increase incentives to improve pupil performance. The worldwide evidence in support of these reforms is mixed, although a majority of studies find mild positive effects.

However, as I’ve argued elsewhere, it is also true that many of the reforms have been watered down due to political opposition – and most systems have consequently suffered from poor design. In most cases, in fact, incentives to improve performance have barely increased at all. And whereas opponents of market and government accountability have argued that the reforms don’t work well, proponents have argued that policymakers simply have not gone far enough to make sure they work better.

This begs the question: what would truly transformative change toward market and government accountability produce? In this paper, economists Douglas Harris and Matthew Larsen provide evidence from New Orleans, which, after the devastation of Hurricane Katrina in August 2005, put in place the most radical market- and government-accountability system ever created in America – in what can be described as an experiment in ‘education shock therapy’.

Prior to Katrina, New Orleans had a traditional neighbourhood-based education system, managed by local school districts, with teachers getting paid according to single salary pay scales decided by union contracts. After Katrina, however, this all changed. The state took control over the school system, eventually turning all schools into charter schools, while all educators were fired. Teacher union contracts were not renewed after they expired, and local control of schools essentially vanished entirely, apart from ensuring per-pupil funding and deciding which schools were to be opened and which ones were to be closed with help from scores in the government accountability system. Attendance zones were abolished, which increased opportunities for school choice considerably. Within just a couple of years, therefore, the system underwent a radical change – the devastation following Katrina simply made it impossible for political opposition to derail the reforms.

What, then, were the results of this shock therapy? Using several difference-in-difference strategies – and a matched comparison group that were subjected to Katrina but not the school reforms (allowing them to take into account disruption due to the hurricane) – the authors find strong positive effects of the reforms. In the beginning, effects were rather mild, but they grew stronger over time as the reforms took effect. Indeed, after seven years, the reforms had generated improvements equivalent to about 30-40 PISA points. These effects are apparent in both the most high-stakes subjects (mathematics and English) as well as in less high-stakes subjects (science and social studies). Also, pupils from different backgrounds all benefited from the reforms, although poor and African American pupils appear to have benefited slightly less than non-poor and white pupils in the first years following Katrina. It is difficult to know, however, whether this difference is due to effect heterogeneity or the fact that poor and African American pupils were hit the hardest by Katrina.

Overall, therefore, the reforms have clearly been instrumental for generating substantive improvements in pupil performance. Certainly, the pre-Katrina system was low performing, limiting the external validity to other poorly performing systems. Similarly, it is unclear whether similar reforms at the country (or state) level would be able to generate the same types of gains; it is always difficult to know whether or not successful reforms can be scaled up without losing some of their successfulness. For example, post-Katrina New Orleans received a lot of outside help from educators, many educated through Teach for America, supporting the reform efforts.

However, an important point is also that the post-Katrina system, unlike many other efforts to pursue government and market accountability, did fundamentally change the overall incentive framework in the education system. The tragedy and devastation of the hurricane meant that political opposition to the reforms were muted, which is normally not the case. The ‘shock therapy’ strategy of reform may be successful but is incredibly difficult to implement politically. So the question is: how could we ever introduce such radical reforms without the tragic devastation of a hurricane?

Gabriel Heller Sahlgren

This comment piece is also the Editor's Pick in the CMRE Monthly Research Digest_02_16. The piece reviews a paper by Douglas N. Harris and Matthew F. Larsen, ‘The Effects of the New Orleans Post-Katrina School Reforms on Student Academic Outcomes’ for the Education Research Alliance for New Orleans (Tulane University), a free copy of which may be downloaded here.

You can download free copies of back issues of the CMRE Monthly Research Digest here.

 to subscribe.