Like many of my colleagues, I got into academia because I wanted to make a difference to the world. Indeed, this is the optimistic aim that I take to work each day. I was fortunate to play a role in an important research project on the use of research in Parliament. Led by UCL STEaPP and the Parliamentary Office of Science and Technology, the ESRC-funded project assessed how research was defined and used in Parliament.
We have produced an exciting report, which was launched today in Parliament. There was an interesting array of speakers, including Sir Mark Walport (now heading UKRI), two Members of Parliament (Dr Roberta Blackman-Woods MP, and Adam Afriyie, MP), Lord Willets, and Professor Tony McEnery (ESRC Research Director). One of the most satisfying things about being an academic is producing outputs that are shared with user communities. The report can be found here, but I want to focus on one key point – namely, the need for an incentive structure in academia that rewards evidence collation and the production of policy relevant recommendations.
Sir Mark Walport concurred with one of the most important findings of the report, which illustrated that individuals in Parliament (MPs, Peers, staff) are more interested in what the body of evidence says, rather than the latest paper. People in Parliament are short of time, and therefore if evidence cannot be communicated to them quickly in a useable form, then it is unlikely to make a difference. It was concerning, therefore, to note that academic research was criticised by many people in Parliament for being hard to access, difficult to understand, poorly presented, too long-winded, and quite frankly irrelevant for current policy questions. From my perspective, it’s unsurprising that academic research is perceived in this way because the incentive structure placed before researchers (i.e. the REF) is poorly thought-out and treats policy impact as an afterthought. Chris Tyler, who co-led the UCL/POST project, talk about this here and we should seriously consider his proposals. Yes, impact is rewarded in the REF, but career progression is still often linked to the ability to produce 3-4* papers, which must be ‘world leading in terms of originality, significance, and rigour’.
‘Originality’ is a huge problem in that definition. Of course, part of academia is about making new discoveries. Yet, as shown by the responses of MPs, staff, and Peers in Parliament, the most useful research for policy-making is not the latest shiny, new paper, but rather an accessible review of the existing body of evidence. A shining example of how to do this is the ‘What Works in Conservation Science‘ initiative by Professor Bill Sutherland at Cambridge (Conservation Evidence). It’s a fantastic innovation, but probably not REF-able. This won’t matter to Bill as he’s already progressed to a senior level, but would an Early Career Researcher be able to get away with doing something so useful? I’ve also got experience, as have many colleagues, in conducting systematic reviews and pulling out a series of key actionable recommendations from these. Now, imagine my surprise in my first REF preparation meeting when I was told that my review papers were practically worthless. Instead, the strategy should be to focus on originality and innovative methodologies. So we are actively being encouraged to produce, as Dan Sarewitz would say, an excess of data, rather than trying to make the best use of data we already have.
As I write on, do keep in mind that policy-makers in Parliament want simple, accessible summaries, not complicated, incomprehensible pieces of writing. This pleased me enormously. In my undergraduate course, I had thought that I wasn’t clever enough to understand the complicated ways in which some (and it’s only some) human geographers write. Each day, however, I am given more ammunition to say that such a style of writing isn’t clever at all. For example, I’ve worked with farmers who just want you to say what you mean, instead of talking in riddles. Dr Roberta Blackman-Woods MP made a similar point as she remembered a recent meeting with researchers in which the evidence was presented in a complex, inaccessible manner. Policy-makers and other users of research really don’t care about the latest new word that you’ve dreamt up (and gosh I came across quite a few long made-up words in my Geography course).
The REF meeting reminded me of an old interview in which an academic accused me of doing consultancy, rather than academia, because my work was too policy relevant. In contrast, he described his work in a much fancier way. It also reminded me of a former colleague who used the phrase ‘policy relevant’ as a criticism on dissertations (because it wasn’t clever enough). In the first case, I didn’t get the job – with hindsight it was probably a poor idea to reply with ‘if you’re happy doing academic research that is theoretical, is published in journals that no one outside of your discipline reads, and makes no difference to the world, then fine, but I’m not. That is not my definition of academia.’ The interview didn’t last much longer, but it was fun at the time.
At the risk of losing my optimistic mindset, this system is madness. To illustrate why, let me tell you about a new paper that I’m writing. It’s for a Geography journal and it was originally aimed at providing tangible advice to developers of new technologies. The paper, however, doesn’t do this any more. After being advised by a former colleague that I needed to ‘make it less useful and more theoretical’ for the intended audience (and more REF-able), I then wrote a far less useful, but more fancy-sounding piece. This is going to be more REF-friendly, and I’ve used some pretty big words in it; but what was the point in making it less useful to the world just to satisfy REF requirements? To the outside viewer, such as the taxpayer, there was no point at all and it isn’t something that I intend to do again anytime soon.
The clear finding of the report is that people in Parliament want accessible summaries of the evidence. They don’t want to be bombarded with the latest single studies, and they aren’t particularly interested if you’ve invented a crazy methodology or new word just to make your work seem more exciting. Rather, they want an impartial overview of the body of evidence about a topic, which provides relevant recommendations. So my advice to the academic community, particularly in the social sciences, is the next time you write a paper, just say what you mean as simply as possible. Don’t just criticise a grant application for not having an innovative methodology or not being theoretically elegant, make sure you also consider whether the project is likely to be useful to the world. A natural science proposal, for example, wouldn’t dream of proposing an innovative methodology if an existing standardised methodology existed, so why the obsession with new methods in social science? Because if it is going to be useful and policy relevant, surely that is more important? At the end of the day, we are all in academia to make a difference to the world.