Teaching practices and cognitive skills: how are they related?

One of the most hotly debated topics in education today concerns the gradual decline of traditional teaching methods, such as lecturing, in favour of more pupil-centred methods, which focus more on group work/discussion. These methods, which have been introduced under different names by different reform movements, are supposed to prepare pupils for a shifting labour market in which reasoning skills are becoming more important. Indeed, pupil-centred methods are today preferred by many educational organisations. Yet it is clear that the evidence base for these methods has been meagre at best. The first issue of this research digest discussed a study that found that test scores, both national and international, declined significantly in Quebec as a result of the province’s introduction of pupil-centred methods en masse. This is not an isolated finding: other research has also found that traditional teaching methods are preferable if we are interested in raising test scores. Of course, this evidence only shows that pupil-centred methods may be harmful, at least in comparison to superior teacher-centred methods, for the type of skills covered by the tests. Interestingly, while teaching methods have changed, the content of most tests has not. It could be the case that traditional teaching methods are good for increasing scores in traditional tests, whereas modern teaching practices are better for other types of skills that are not measured by those tests. One of the key rationales for utilising pupil-centred methods is indeed to promote reasoning skills over factual knowledge and basic problem-solving skills – but most tests focus on the latter. In a new paper, economist Jan Bietenback finds evidence the above holds true. The author analyses TIMSS test scores in mathematics and science among American eighth graders in 2007. By decomposing scores in three different domains, he is able to separate the effects of different teaching practices on ‘knowing’, ‘applying’, and ‘reasoning’ domains, with the weight being about 35%, 41%, and 24% respectively in mathematics, and 37%, 41%, and 22% respectively in science. Importantly, it is the last domain that measures skills that many advocates of modern teaching practices seek to promote. The author analyses how differences in teaching methods that pupils experience in science and mathematics are related to test-score differences in these subjects, which means that he can hold constant unobservable pupil characteristics, such as intelligence, maturity etc., that affect their results equally in science and mathematics. Interestingly, he finds that traditional teaching methods are clearly best for raising achievement in the two ‘traditional knowledge’ domains: a 100% increase in traditional teaching methods raises achievement in those domains by fully 0.42 standard deviation in the ‘knowing’ domain, and by 0.36 standard deviations in the ‘applying’ domain. However, modern teaching practices are best for raising achievement in the ‘reasoning’ domain: a 100% increase in modern teaching practices increases achievement by 0.24 standard deviations in this domain. While the differences are only statistically significant in the first two domains, this indicates that the different teaching practices are good for different things. The author also analyses data from 16 education systems pooled, and finds very similar effects, with the differences now being statistically significant also in the reasoning domain. It should be noted that the research methods utilised are unable to control for all potential bias. For example, unobserved teacher traits could correlate with the choice of teacher method and domain focus in the classroom. If this is the case, and the unobserved teacher trait affects pupil performance, the results are biased. While more research clearly is needed before we draw too strong conclusions, these are nevertheless important findings. First, they point to a potentially disturbing trade-off between different types of knowledge and skills. While reasoning skills may increase with modern teaching practices, traditional skills may be reduced significantly at the same time. Second, it indicates that an important reason modern teaching methods often fail to raise test scores is simply that their performance are measured by results in traditional tests. For example, only about 20-25% of the questions in TIMSS focus on such reasoning skills. If tests were devised differently, therefore, results may differ.* In general, given the trade-off, it is difficult to draw clear policy conclusions – it depends on what type of knowledge and skills we are mostly interested in promoting. While reasoning skills are likely to grow in importance for success in the labour market, one could also argue that traditional skills will remain key for many jobs. For example, a thorough understanding and knowledge of algebra is important for further education in mathematically intensive subjects and vocations at the university level, such as engineering and computer science, which are often emphasised as crucial for countries’ prosperity. Future work should study the link between different test score domains and teaching practices with earnings in order to investigate this further. It is therefore far from clear that traditional skills should be abandoned wholesale. The first step must be to determine what skills and knowledge we want to promote. In this process, competition is likely to play an important part – strong school autonomy in teaching practices and curricula, in combination with more school choice, is a way to spur that competition. In combination with a rigorous research programme to study the effects in the labour market, such an approach is likely to generate a much better understanding of these issues. ____ *On the other hand, results reported in the first issue of this year’s research digest indicate that traditional teaching methods are also preferable for raising PISA test scores, which are supposed to focus more on reasoning skills than traditional skills. Gabriel H. Sahlgren This comment piece first appears as the Editor's Pick in CMRE Research Digest (September 2014). The piece reviews a paper by economist Jan Bietenback, 'Teaching Practices and Cognitive Skills' appearing in the March 2014 issue of Labour Economics. A free copy of the working paper may be downloaded here. You can download free copies of back issues of the CMRE Monthly Research Digest here. to subscribe.