Research Excellence Framework (REF)
In REF2014 Loughborough was ranked a top ten research University in the Times Higher Education (THE) intensity rankings. We are now preparing for our REF2021 submission.
What is the REF?
The Research Excellence Framework assesses the quality of research in UK Higher Education Institutions. The next REF will be assessed in 2021, across 34 subject based Units of Assessment (UoA). The REF informs the selective annual allocation of funding for the period between exercises (c. 6 years), awarded by the four UK higher education bodies. The assessment provides accountability for public investment in research and produces evidence of the benefits of this research in the form of case studies.
In REF2021 three main areas will be independently assessed by UoA Panels; analysis of outputs (60%), impact (25%) and research environment (15%). In 2014, quality ratings for all three sub-sections led to an Overall Quality profile for each UoA submitted. Loughborough University made 18 submissions to 17 UoAs. For further detail on how we performed in 2014 see the REF2014 website.
REF Code of Practice
All higher education institutions who intend to make a submission to the 2021 Research Excellence Framework (REF) are required to develop, document and apply a Code of Practice on the fair and transparent identification of staff and selection of outputs for REF submission.
Loughborough University’s Code of Practice was drafted by a working group chaired by the Pro-Vice Chancellor (Research) and consisting of academic staff, Human Resources, Loughborough UCU, the Planning Team. A consultation on the content of the Code of Practice ran between 27 March and 1 May 2019 and a summary of the main feedback and actions taken are available. Our Code of Practice was approved by Research England on 16 August 2019 without changes. Our approach to REF is regularly refined in line with its broad principles and in response to REF FAQs and further guidance. Where the details of processes have changed, they are being communicated to staff via both existing channels as outlined in the Code of Practice itself, but also updated on the Code of Practice using track-changes to clearly highlight the changes made.
Please see our Data Collection Statements for both Loughborough University staff and non-staff.
Following the 2014 Research Excellence Framework, Lord Stern performed an independent review of the exercise to which Loughborough University responded. Following Lord Stern’s report, HEFCE published a consultation on some proposed changes to the REF.
The key decisions include:
- All staff with a significant responsibility for research should be submitted.
- The average number of outputs per member of staff submitted should be 2.5 with a minimum of 0 (where staff have individual circumstances) and a maximum of 5.
- This may include outputs of staff who no longer work at Loughborough University as long as they were demonstrably generated here.
- Each submission should include a total of two case studies (up to 19.99 FTE), plus one additional case study for up to 15 FTE….
Sub-panel Chairs were announced in December 2017.
Self-help resources for Impact Case Study authors
This section contains a range of detailed self-help resources for Impact Case Study authors, and those supporting them, offering practical guidance and tips in relation to defining and articulating your impact; evidencing and measuring your impact; writing your Impact Case Study; and reviewing your Impact Case Study.
REF Support Structures at Loughborough
Loughborough University Submission to REF (LUSTRE)
LUSTRE meetings are scheduled for the Spring 2019, Winter 2019-20 and Spring 2020. All REF UoA Teams will receive guidance on their progress from internal and external advisors including PVC(R), PVC(E), REF Panel reps, senior staff, Research Policy and Planning, Planning Team. Chair: PVC(R). Contact: Elizabeth Gadd & Angela Crawford, Research Office. Email: email@example.com
Minutes of previous LUSTRE meetings, redacted to avoid identifying individuals, are available here.
Code of Practice Working Group
The Code of Practice Working Group is a short-term working group established to steer, design and implement the University's REF Code of Practice around the identification of staff with significant responsibility for research, those that qualify as independent researchers and the selection of outputs. It is attended by representatives from academic staff, HR, Loughborough UCU, Research Office. Chair: APVC(REF). Contact: Elizabeth Gadd, Research Office. Email: firstname.lastname@example.org
Minutes of previous Code of Practice Working Group meetings are available here.
REF Panel Member Advisory Group
The REF Panel Member Advisory Group consists of Loughborough staff who sit on, or secretary, REF 2021 panels. It meets occassionally to guide the university (within the bounds of confidentiality imposed by the REF) on its submission. Chair: PVC(R). Contact: Angela Crawford, Research Office. Email: email@example.com
Open Research Working Group (ORWG) [Was Open Access Advisory Working Group]
The Open Research Working Group exists to oversee the implementation of the Open Research Position Statement, and with it the management and operation of the Research Repository and LUPIN (LU Publication Information) and to advise Research Committee on open research matters. It meets quarterly and is attended by representatives from academic staff, library staff, Research Office and IT Services. Chair: Professor Camilla Gilmore. Contact: Katie Appleton and Naomi Dungworth, Library. Email: Repository@lboro.ac.uk
REF OA Action Group
This is an informal group of Library and Research Office staff that meet regularly to discuss operational matters relating to Open Access, the Research Repository and LUPIN (LU Publication Information). Contact: firstname.lastname@example.org or email@example.com
REF UoA Teams
Each Unit of Assessment to which Loughborough University intends to submit is supported by a team, headed up by a UoA Lead. Each additional School submitting to a UoA will also have it's own School Lead. UoA Teams are responsible for writing their environment statement, identifying independent researchers, and selecting outputs and case studies for submission in line with Loughborough University's Code of Practice. A list of the UoA Leads and their contact details are below.
|UoA Name||Main Panel||UoA number||Overall Lead|
|Public Health, Health Services and Primary Care||A||2||Paula Griffiths|
|Mathematical Sciences||B||10||Andy Archer|
|Computer Science and Informatics||B||11||Qinggang Meng|
|Architecture, Built Environment and Planning||C||13||Malcolm Cook|
|Geography and Environmental Studies||C||14||Louise Holt|
|Business and Management Studies||C||17||M.N.Ravishankar|
|Politics and International Studies||C||19||Caroline Kennedy-Pipe|
|Social Work and Social Policy||C||20||Steve Case|
|Sport and Exercise Sciences, Leisure and Tourism||C||24||David Stensel|
|English Language and Literature||D||27||Michael Wilson|
|Art and Design: History, Practice and Theory||D||32||Richard Bibb|
|Communication, Cultural & Media Studies, Library & Information Management||D||34|| John Downey
Where do I go for help with Vertigo Ventures Impact Tracker (VVIT)?
I believe I qualify for a reduction in outputs due to my circumstances, what should I do?
The UK funding bodies are committed to supporting and promoting equality and diversity in research careers. As part of this commitment, measures have been put in place to recognise the effect that individuals’ circumstances may have on research productivity, resulting in a reduction in outputs. Please see paragraph 160 of the REF Guidance on Submissions for a list of circumstances that apply. Loughborough University's Code of Practice Working Group have developed a form through which staff can voluntarily declare any such circumstances. Please click here to access the form. If you have any questions about this, please either contact your UoA Team Lead or the Research Policy Team on firstname.lastname@example.org.
Can we submit outputs written by individuals before they became 'Category A' eligible? (e.g., former PhD students or University Teachers)
If the member of staff is category A eligible (i.e. has SRR and independence) on the census date, then they and their associated outputs should be returned to REF 2021. Current members of staff will need to be Cat A eligible on the census date, but it is not a requirement for them to have been Cat A eligible when their outputs were first made publicly available (n.b. that outputs should be published within the publication period and meet all output eligibility criteria).
Outputs published while a person was a PhD student can be submitted providing the person is category A eligible on the census date. As the guidance says, the thesis, dissertation or other materials submitted for the PhD are not eligible outputs, but published research outputs are (e.g. articles published during a PhD).
If a member of staff was Cat. A eligible at one point during the REF period, but is not on the census date, then they cannot be included in the submitted staff or output pool calculation, and none of their outputs would be eligible unless they are submitted under the scope of former staff: former staff would need to have been Cat. A eligible at the time that the output was first made publicly available.
How are outputs assigned to individuals for REF?
The purpose of REF 2021 is to assess the research excellence of universities and units within those universities, rather than individuals. To this end, the expectation is that all RTE staff are submitted rather than a selection. The total number of outputs submitted by each UoA must equal 2.5 times the summed full-time equivalent (FTE) of the unit’s submitted staff, with a minimum of one and a maximum of five outputs per person. Thus, whilst outputs will need to be allocated to individuals to demonstrate that the minimum of one output per person is met, the matter of who each output is associated with makes no difference to REF or to the University. Indeed, Loughborough University's Code of Practice has made it clear that whether an individual has been submitted to the REF, or the number of REF outputs associated with an individual, will not be used as a criterion for recruitment, probation, reward or promotion. Whilst the REF will publish a list of outputs submitted to the REF by each University, they will not display the individual with whom each output was associated. To this end, once the requirement for a minimum of one output per person has been met, the remaining outputs will be allocated to individuals in such a way as to maximise our return.
I am named on a good output, but it has a large number of co-authors, can I still submit it?
Yes, as long as you made a ‘substantial research contribution’ to it. This may be audited. In UoAs 2 (Public Health) and 9 (Physics), if an output has over 15 co-authors, and the individual to whom it is attributed is not the lead or corresponding author, a statement should be supplied outlining the individual’s contribution to that output. Guidance on supporting statements has been circulated to UoA Leads, and multi-authored output guidance is also available in the REF Panel Criteria and Working Methods paras. 216-236
Can my reviewer use the peer review scores resulting from REF output scoring processes in my PDR?
Output scores resulting from REF-based peer review processes can form part of the discussion of output quality in your PDR. However, whether, and how many of, those scores hit the threshold for inclusion in the REF submission, should not be under discussion.
I’ve not been told the REF scores given to my outputs, how can I find these out?
Your UoA Lead and/or ADR can supply you with the scores currently allocated to your outputs, but not the scores of others. However, it should be borne in mind that REF output scores may not be static; they may continue to be calibrated based on internal and external advice. The main purpose of scoring is to rank outputs so that we can select those we think are most likely to give Loughborough the best outcome.
I don’t agree with the scores that have been allocated to my outputs, can I raise this in my PDR?
Output quality scores can make an important contribution to your career development plans. If you feel you would like to discuss the scores which have been attached to your output(s) together with any associated review comments, you can do this in your PDR. For REF we use output quality scores to help decide which outputs to submit for assessment but this is not a matter for PDR. While output scoring is not an exact science, it can still be a valuable basis for PDR discussion of output quality in terms of visibility, significance, rigour and impact.
I’ve heard that up to five of my outputs are currently being considered for REF submission, can I include this on my PDR form?
Loughborough’s REF submission will not be finalised until November 2020 and the list of outputs for each UoA will continue to fluctuate until submission. Rather than discuss the number of outputs currently hitting the threshold for submission, which may change, any discussion (or written comments) around outputs being considered for REF would more usefully be focussed around the peer review feedback you have received. It is important to remember that whilst PDR is an opportunity to celebrate past achievements, it is also forward-looking and should be seen as an opportunity to discuss how you might act on reviewer feedback to improve your outputs in the future.