Everybody agrees that teachers are important for improving academic outcomes. But how important are they for improving non-cognitive outcomes? There’s surprisingly little research analysing this issue. This is a significant gap since research indicates that test scores do not pick up such outcomes – such as adaptability, self-restraint, and motivation – which have been found to be important for longer-term life outcomes. This implies that measures of teacher effectiveness likely to function better if they could also isolate teachers’ impact on non-cognitive outcomes, rather than merely on academic results.
In a recent paper, 'What Do Test Scores Miss? The Importance of Teacher Effects on Non-Test Score Outcomes
', Kirabo Jackson sheds new light on this issue. Exploiting administrative data from secondary-school pupils in North Carolina, he uses a number of behavioural outcomes as proxies for pupils’ non-cognitive skills – suspensions, absences, course marks in ninth grade, and whether they enrolled in tenth grade on time. By isolating the variation in these measures that is unrelated to test scores, he is then able to create a teacher-effectiveness index that solely measures non-cognitive outcomes.
The author then also creates a regular value-added metric based on test scores. Importantly, there is only a weak correlation between the different measures of teacher effectiveness, suggesting that they indeed capture different skills. He then sets out to analyse how the different metrics predict future pupil outcomes.
The results are fascinating. When using the regular value-added metric, a one standard deviation increase in teacher effectiveness predicts a 0.14 percentage-point higher likelihood of graduating from high school. However, when including teacher effectiveness in raising behavioural outcomes, this impact decreases slightly to 0.11 percentage points – while a one standard deviation increase in non-cognitive teacher effectiveness boosts this likelihood by 0.78 percentage points further. The full predictive value of teacher effectiveness on the likelihood of graduating increases by fully 249% when also including the non-cognitive measure. Results are similar when analysing the likelihood of dropping out, that pupils take the SAT university admissions test, their high-school GPA, and their self-proclaimed plans to attend four-year colleges.
Certainly, more research is necessary, especially to verify that these effects are similar in other institutional contexts. It is also worth noting that many of the behavioural outcomes can be manipulated, suggesting it may be difficult to use them in accountability systems. Other alternative metrics must therefore have to be developed and tested for this purpose.
Furthermore, while the research shows that teachers have important effects on non-cognitive outcomes, which in turn affect pupils’ futures, it is silent on what teacher characteristics that predict their effectiveness in this respect. Previous research searching for observable characteristics explaining test-score based value added has been relatively futile – but it is possible that researchers may be more successful when it comes to predicting teachers’ non-cognitive effectiveness.
Regardless, the paper strongly suggests that teachers’ contribution to the future of pupils go far beyond their improving academic outcomes, which is something many people have believed but no one has thus far been able to show rigorously. This makes it an interesting and important contribution – both to the literature and to policymakers worldwide.
Gabriel Heller Sahlgren
This comment piece is also the Editor's Pick in the CMRE Monthly Research Digest_04_16. The piece reviews a paper by C. Kirabo Jackson, 'What Do Test Scores Miss? The Importance of Teacher Effects on Non-Test Score Outcomes' published as NBER Working Paper No. 22226, a free copy of which may be downloaded here.
You can download free copies of back issues of the CMRE Monthly Research Digest here.