Back to Top

Exam boards competing on standards?

Michael Gove has framed his proposals for franchising 14-16 exams provision on a one board per subject basis as part of a wider effort to address the problem of exam boards competing on standards. While there is certainly a case for reform or disposal of the comparability framework (which I think is what the Minister is driving at), his rationale for doing so is unhelpful in the task of increasing public understanding of where the problems lie. The popular perception, validated by the Secretary of State in his choice of the vocabulary of ‘competitive dumbing down’, is that by its very nature, competition has led the boards to develop qualifications that are easier to do well in, in order to gain market share. This obviously goes well beyond the allegations, made by the Daily Telegraph and others, that guidance given by examiners at teacher support seminars routinely includes information about exams which compromises their integrity (which it should be noted, have subsequently been found to apply to just six GCSE papers of a hundred prepared for 2012 examination). The charge here is that exam boards have been deliberately lowering standards either by requiring less and less of students, year on year, in respect of the level of skills, knowledge or understanding they are expected to demonstrate, or by exaggerating the achievement of students by awarding progressively higher grades to comparable exam submissions from one year to the next (a practice usually referred to as ‘grade inflation’), in an effort to win custom. To many, particularly (but not exclusively) to those on the left of the debate, this might seem at first pass a plausible enough account of what has been going on, but does it stand up to scrutiny? Would examining boards deliberately pursue a business strategy that would directly impact their credibility as guardians of the standard of our qualifications? How would it be in their interests, either individually or collectively, to undermine the currency of their qualifications by lowering those standards? Granted, they might be tempted to do so for short-term gain, but wouldn’t you expect in such an industry that there might be mechanisms in place to deter such behaviour, and to safeguard the investment that each have made in their brands and what they stand for? Apparently not, but nevertheless, they do exist; and given that they are the mechanisms on which the system has largely relied to quality assure national examinations, it is important to understand how these processes work. First, projections of levels of attainment at key grade boundaries are made for each paper on the basis of how the prior attainment of students entered for the subject specification compares to those entered across all boards in the previous year. Results that deviate from the prediction by more than 1% are subject to rigorous investigation by each of the boards individually, by their competitors, and by Ofqual before they are issued – which intervenes to ask boards to change their boundaries, to bring results into line with others, if necessary. This ensures that even the subtlest of attempts to lower the standard for competitive advantage is picked up before ever results are issued. Secondly, after the fact, screening is then carried out every Autumn by the Joint Council for Qualifications (JCQ) to look at how closely aligned the results for each subject specification were to national levels of attainment for that subject. The results of this screening process, and of a similar exercise conducted at five year intervals by Ofqual, are then used to inform the setting of grade boundaries in subsequent years. Finally, when results are published, they are aggregated to disable comparison. As a further safeguard, JCQ protocols expressly forbid the use of performance data for marketing purposes. So while very occasionally examples do emerge of ill-informed marketing that suggests otherwise, it really isn’t possible for exam boards to differentiate themselves from their competitors in this way. The boasts of examiners in the context of training seminars should be viewed in a similar light. Claims suggesting that their boards set easier exams or mark more generously cannot be substantiated; they merely play (foolishly) to a perception. The reality is that standards are set and comparability monitored in England across committees and exam boards, in a distributed fashion. In the opinion of the author of The Centre for Education Research and Policy (CERP)’s submission to the Education Select Committee (whose particularly helpful explanation of the above processes informed my summary above), there is no way that a single board could get around these mechanisms to secure a competitive advantage. On the contrary, doing so would require a breath-taking degree of collusion: only ‘if all awarding bodies were to, in one year, reduce to the exact same extent the content of their specifications and assessments’ might this pass by these mechanisms undetected, but of course, that would be impossible theoretically as well as practically, given the sheer numbers of people involved. So much, then, for the charge of deliberate compromise. What of teachers’ perceptions of whether one board’s specification might be more straight-forward, or marking more positive, than another’s? One indicator of this might be data suggesting that schools are regularly switching between boards. Data provided to the Education Select Committee enquiry by Ofqual (Annex A, Figure 2) however indicates that overall market share has remained relatively stable since 2006. Chief Executive, Glenys Stacey, told the enquiry that there had been little research done on whether schools are regularly switching, or of the rationale for doing so (First Report, para 132), an observation also made by the NAHT (para 134). CERP were only able to cite a single study, by Malacova and Bell (2006), looking at whether centres switching English GCSE specifications between 2000 and 2003 did so on the basis of data indicating a higher percentage of candidates might thereby achieve grade C. That study found there to be insufficient evidence for the assumption. The Select Committee identified from inter-board statistics published by the Joint Council for Qualifications (JCQ) that the most significant shifts in market share occur, as one would expect, at moments of syllabus revision. This would seem a natural point for heads of department to consider their options. All of which leaves one wondering whether competition is in fact contributing to the standards problem as is claimed, and therefore whether Michael Gove’s proposal for introducing franchising as the solution is well grounded. While there is a broad consensus in the assessment community that ‘grade inflation’ probably does exist (enabled by the above outlined statistical predictions mechanism allowing for increases of up to a percentage point each year), the phenomenon cannot be said to be the result of any competition effect. Rather, grade inflation might more accurately be described as the result of an accumulation of cultural assumptions and practices which have been shaped more by political priorities and pressures than anything else (more on this in a subsequent post). As Rod Bristow, President of Pearson UK, wryly commented in an interview with The Times recently, if anything we’ve had ‘a system in which the exam boards were too driven by the regulator and too compliant’ with its standards, not the reverse. The problem is not that measurement of comparability has gone badly wrong, but that we have simply met the limitations of that science. Likewise, the problem is not that the delivery framework for national qualifications has encouraged the wrong kind of competition, but that it cannot accommodate efforts to raise the standard. Against the background of his early promotion of alternative private qualifications like the IGCSE and International Baccalaureate, and reforms to tackle the inflationary effects of too generous equivalencies in the Qualification and Curriculum Framework (QCF), I think the Minister has realised this, but doesn’t have the courage to go all the way and trust the market. The problem is that unless he does so, we will end up in a worse position overall than when he started, because experience has taught us that efforts to raise the standard within the national qualifications framework inevitably end up disadvantaging some types of learners. In narrowing the focus of the curriculum through the EBacc; in the introduction of a two-tier O level/CSE to enable greater differentiation by ability at the middle to high end; through limiting the proportion of the timetable that can be made available for vocational learning; through excluding lower level ‘step-ladder’ qualifications; and in ending modular assessment, Gove's reforms will undoubtedly make it much harder for some students to demonstrate what they know, understand and can do. Of course we must have high expectations of all, but that is different from requiring the same of all. If Gove succeeds in restoring ‘rigour’, he will do so only at the expense of ‘access’, in the process sweeping away undoubted gains of recent years in efforts to de-mystify what examiners are looking for, broaden the choice of qualifications, aid progression among dis-engaged students, and recognise the formative role of assessment in learning. If the national curriculum and national qualifications were to be scrapped, on the other hand, we’d lose the floor standard, but that would liberate assessment providers to invest in alternatives like the IGSCE and Pre-U, and to develop new syllabuses and qualifications designed to better stretch students of academic ability in preparation for university entrance. It would also incline them, furthermore, to tighter conversation with vocational learning providers and employers, to develop programmes of learning and qualifications that would genuinely aid career progression for those heading in other directions. They would be incentivised to provide a range of learning opportunities appropriate to the diverse needs of the labour market, which would in turn lead to real and valuable employment opportunities. And because information about which qualifications were in highest demand would be important for parents and students, university and employer-engaged independent qualifications reviews would become an important resource for career and higher education advisors, replacing the finger-in-wind judgements of the QCF. With schools freed to provide those qualifications for which they believe students are best-suited and held to account for those decisions, not by government, but by students and parents, teacher/school accountability might at last be given the opportunity to evolve beyond the crude use of assessment data to more contextually nuanced means.

Blog Category: 
About the author
James Croft is Director of the Centre for Market Reform of Education and the co-author of the CMRE discussion paper 'When Qualifications Fail: Reforming 14-19 assessment' (2012) and previously of 'Profit-making free schools' (2011).