The REF: is there another way?

In March, staff in Chartered ABS member schools pressed the big red “submit” button for the Research Excellence Framework (REF). This national census allocates billions of pounds of public money in support of research activities, and the REF2021 outcomes will bear real consequences for individual staff, for schools and for universities. Those outcomes won’t arrive quickly because REF panellists will spend the next several months rating thousands of outputs, impact cases and environment statements before calibrating, cross-checking and finally releasing results this time next year. Whilst we’re waiting, the business school community has the chance to engage in discussions about what we’d like to see happen next. It is time for some honest reflection and for our community to get vocal.

There are long-running debates about the relative merits of peer-review and metric-based approaches for research. No approach is perfect but the time and attention absorbed by the REF exercise has grown out of all proportion to the benefits it delivers. As academics we are well used to designing assessments that adequately measure learning outcomes but do not absorb excessive amounts of time for our students, or for us.

Time spent on the REF occurs on multiple levels. At the highest level, the REF panels themselves spend time reading, rating and reporting. At university-level, our institutions have developed apparatus, roles and meeting structures to predict REF performance across all of their disciplines. Further down the hierarchy, individual Units of Assessment have teams which pulled together data in the early stages of the assessment cycle and progressively turned their attention to internal ratings, mock exercises and preparing draft submissions and narratives. Finally, within UoAs, individual research groupings and individual researchers obsess over how to respond to interim feedback from within their institution. The time taken is inordinate. Worse, it is a burden that falls disproportionately on our most talented researchers because their individual excellence means that their ability to discern between good and less good research is prized.

When you layer the opportunity cost of REF with equivalent exercises in teaching and in knowledge exchange, it is hard to escape the conclusion that the scrutiny of performance is taking up valuable time that should actually be spent on delivery. Elsewhere I have written about the pernicious impact of over-assessment and argued for a dramatically simpler, metric driven approach that uses a combination of data that already exist or are easily curated. Metrics spanning research, knowledge exchange and teaching at disciplinary level could then be aggregated up to provide a profile of the very different types of universities that operate in the UK. We shouldn’t shirk those differences but rather we should celebrate the fact that they reflect different missions, histories, scale and choices across universities.

The current REF, TEF and KEF arrangements are imperfect. They take too long, absorb too much time from our most talented academics and run over assessment cycles that outlive most leadership tenures, meaning poor outcomes are always someone else’s fault. An integrative, metric-driven system would also be imperfect but quicker. It is worth noting that there is little likelihood of consensus on how the sector is evaluated. In that context, a courageous next step would be to speak openly about the choices we face. In research terms, I can see three options.

Option A: scrap the REF exercise and give out research funds on a per capita basis. Automating the allocation of public funds in this way would be truly radical and might be sold as a form of levelling up.

Option B: admit it is too hard to get consensus on a revised REF format and therefore commit to turning the handle again perhaps with minor tweaks such as reweighting impact or prioritising work that connects to sustainability, inclusion, economic recovery, or some other laudable policy imperative.

Option C: acknowledge re-reading research which has already been published adds limited value and adopt a lighter touch, metric driven approach.

I really like Option A but suspect it is unworkable. Option B is the path of least resistance but that doesn’t make it a good outcome. Option C is therefore the winner by default as the least bad way forward. And, if you’re willing to be radical with the REF, you might as well adopt a similar template for TEF and KEF to create a single, integrative profile at discipline level for each university.

Such a move wouldn’t be universally popular, nor would it be perfect. However, it strikes me as being less bad than the current arrangement and represents a bold move to redirect academic time away from evaluating and accounting for the doing of research, teaching and knowledge exchange and back to the core purpose of actually doing these vital academic tasks.