School Vouchers and Student Achievement: the Louisiana Scholarship Program isn’t working

In the past decades, an increasing number of school voucher programmes have been implemented in the US. Geared toward low-income households, these programmes allow participants to use vouchers toward tuition fees in private schools. Since the schemes generally have more applicants than places, lotteries have often been used as a tiebreak device. This means that one can compare like for like when studying the effects of attending private schools using the vouchers. Such studies have generally found no or small positive effects on test scores, but larger positive effects on graduation rates and college enrolment (at least among certain sub-groups). There has been little evidence of negative effects.

Until now. In a new study, Atik Abdulkadiroglu, Parag A. Pathak, and Christopher R. Walters analyse the Louisiana Scholarship Program, a targeted voucher scheme providing funds for poor children currently enrolled in low-performing state schools – graded “C” or lower in the state accountability programme – to opt out of the state system and attend eligible private schools. If the number of applicants exceeds the number of places at any given school, lotteries determine the final pupil allocation. The researchers exploit these lotteries to obtain random variation in private school attendance, which allows them to study the effects on achievement in the first year after the programme was rolled out over the entire state.

The authors find large negative effects. After one year, pupils who received the voucher performed the equivalent of 41 PISA points lower in mathematics, 33 PISA points lower in social studies, 26 PISA points lower in science, and 8 PISA points lower in reading. They are also between 24 to 50 per cent more likely to fail. Pupils from different socio-economic backgrounds and locations are similarly hurt by private school attendance, but younger pupils are somewhat more negatively affected. These are large negative effects and strongly suggest the programme, as currently designed, does not work well. Recent findings also suggest that these negative effects persist after two years of private school attendance, although they appear to be less distinct in reading.

What can explain this large negative impact? Pupils who lost the lottery did not attend worse state schools than the Louisiana average, so this is unlikely to be the issue. However, the authors find that private schools that chose to participate in the programme are more likely to have significant downward trends in enrolment prior to entering the program – compared to those that did not participate – indicating that private schools with pre-existing problems were the ones that joined it. However, the authors find that the negative impact of private school attendance is not generally larger in schools with more sharp enrolment declines prior to the start of the programme, compared with other participating private schools.

Still, over two-thirds of private schools chose not to participate and those schools had more favourable enrolment trends on average. Why did not most private schools choose to participate? One reason may have been overregulation – participating schools can’t charge fees on top of the voucher, select pupils, and must administer the state-wide test. These features may have induced most private schools to opt out of the programme – leaving the poorly performing schools to participate – and there is survey evidence in favour of that argument. If so, the lack of high-quality private schools in the programme may explain the negative effect estimated in the study.

Personally, I’m not convinced by this argument. The state schools that formed the comparison group were also poorly performing and there is no evidence that the participating private schools were really bad schools. Overall, in fact, I find it implausible that school quality could explain the large negative impact in just one year. Another reason suggested is a potential transition effect for the private schools that had to adjust to a new type of test they haven’t administered before. But the second-year negative effects do not generally offer support for this story.

Overall, then, it is difficult to provide a clear-cut reason why the Louisiana programme has worked so poorly in the short term. Before this issue has been investigated more rigorously, it’s impossible to prescribe what should be done to improve it. But one thing is certain: without the randomised trial, enabled by lotteries, we would have no way of knowing it needed to improve in the first place.

Gabriel Heller Sahlgren

This comment piece is also the Editor's Pick in the CMRE Monthly Research Digest_02_16. The piece reviews a paper by Atik Abdulkadiroglu, Parag A. Pathak, and Christopher R. Walters, ‘School Vouchers and Student Achievement: First-Year Evidence from the Louisiana Scholarship Program’, NBER Working Paper No. 21839, a free copy of which may be downloaded here.

You can download free copies of back issues of the CMRE Monthly Research Digest here.

 to subscribe.