New economic research into exam boards finds no ‘dumbing down’ of standards, but system could do better

A new report published today by CMRE – Who’s to produce and who’s to choose? Assessing the future of the qualifications and assessment market – finds no evidence that choice and competition have led to a decline in the standards of national qualifications.

Download a .pdf copy of the research report here. 

The report argues that there is therefore little reason to support the formation of a single government exam board – which it judges likely to increase costs without any gains in relation to quality improvement – while at the same time decreasing the potential for innovation, and increasing exposure to system failure.

The report argues that rather than being the result of any ‘race for the bottom’, with exam boards dumbing down their qualifications and inflating grades to win market share, incentives for schools to choose what they perceive to be easier qualifications are mostly a product of the equivalency framework and the way the value of qualifications are weighted in school league tables.

The equivalency framework is designed to assure stakeholders of the comparability of different qualifications and versions of the same qualification. This effectively makes it more or less impossible for exam boards to brand themselves as producing higher-standard specifications in the same qualification, and thereby to compete on quality. But more than this, the framework forces different types of national qualifications – academic and vocational – into one measure of school performance for the purposes of computing aggregate league table scores – and in so doing incentivises efforts to game the system by picking qualifications that are perceived to be easier to pass.

The report finds that these underlying issues would largely remain in a franchising or procurement system, advocated by some as a solution to the perceived problems of the present user-choice based model.

The report makes a number of recommendations for how the present qualifications and assessment market can be better oriented to quality:

• First, the accreditation framework should be less prescriptive in its attempts to ensure comparability between different qualifications, subjects, and specifications, and instead be based on meeting specified minimum standards. This would mean that boards could offer alternatives based on higher, but not lower, quality and standards. This would generate better matching, and allow exam boards to brand themselves on the standards and quality provided. End-users would then be in a better position and have stronger incentives to openly differentiate between different providers and their products.
• Second, to enable better assessment of the difference between different providers and their products, the report supports a more data-driven approach than stipulating what students have to know and accomplish to achieve different grades. Establishing pupil outcomes at university or in the labour market according to the different qualifications and specifications they prepared for would be a far more effective way of assessing their value.
• Third, it would also be beneficial to introduce a benchmarking test focusing on competencies rather than subject knowledge, in order to provide an independent means of gauging improvements in the quality of pupils' learning. Similar to the national reference test at year 11 to be introduced from 2017, this would help schools, universities and employers to compare the relative value of different qualifications and specifications. The results would not be included in league tables, so would be less likely to be compromised by attempts to game the system.

Other recommendations include that:

• Ofqual should provide information to schools and end-users regarding differences between qualifications, subjects, and versions to ameliorate the potential for providers specialist knowledge to disadvantage these stakeholders in the market. It should also commission, or undertake, research to better establish their relative value.
• Reflecting this alteration in the remit of the regulator, school league tables should be reformed. A number of options are presented to this end, including simply publishing results separately for different qualifications in the same subject area, and different versions of the same, for all subject qualifications – which would decrease incentives for schools to take easy options.

In the event that the government decides to move towards a franchise system without user choice in spite of the report’s recommendation, a number of principles are offered upon which it is recommended such a system should be built.

Download a .pdf copy of the research report here. 

Download a .pdf copy of the Executive Summary here.