Loughborough University
Leicestershire, UK
LE11 3TU
+44 (0)1509 263171
Loughborough University

News and events

Latest news from Loughborough University

10 March 2010 | PR 10/44

New research identifies bias in journal ratings

A trusted system used to measure the quality of academic research produced at many of the world’s leading universities is highly problematic according a new study.

Professor Kevin Daniels of Loughborough University’s Business School was part of a team of organisational psychologists, that also included experts at the universities of Exeter and Leeds, to undertake the study. The group looked at how 168 business and management academics rank the specialist journals in which they publish research and found widespread bias is inherent in the system.

The study, ‘Experts judgements of management journal quality: the role of identity concerns in a contested domain’, raises questions about the role of journal lists as a measure of academic excellence.

Based on a well-established body of knowledge in social psychology known as “social identity theory”, the team predicted and found that subject experts’ evaluations of research in business and management are swayed heavily by personal biases.

Dr Kim Peters of the University of Exeter’s School of Psychology, who led the study, said: “Academics rated the quality of journals more favourably when they had published more papers in that journal, when they were a member of the journal’s board, when the journal reflected their academic interests and their geographical location. We found a bias from start to finish.”

Professor Gerard Hodgkinson of Leeds University Business School, who co-authored the study. said: “Our findings turned out entirely as expected on the basis of our theoretical predictions.  For instance, a qualitatively-oriented scholar with a background in sociology will tend to rate journals that favour sociologically-oriented and qualitative studies more highly than a colleague with a background in psychology and a leaning toward quantitative methods.

“There is a rational explanation for the pattern of findings we observed - such judgements serve to bolster the experts’ sense of selfhood and belonging.  Personal and social identity concerns are the primary driving force behind how we evaluate research quality, but our judgements are also constrained to some extent by the objective reality of the context in which they occur.”

Although the study focused on how business and management journals are rated, the authors stress that the approach they used means the results could be extrapolated across other academic disciplines and fields of study. 

Loughborough co-author Professor Kevin Daniels said: "The study raises serious questions concerning the use of journal rankings in judging some aspects of research performance.

"At the very least, compilers of such lists may need to make their methods as transparent and as rigorous as might be expected in a peer-reviewed article, so that the lists themselves can be judged. However, a serious rethink may be needed on alternative or supplementary approaches that are rigorous, fair and also seen to be rigorous and fair."

−ENDS−

For all media enquiries contact:

Amanda Overend
Senior PR Officer
Loughborough University
T : 01509 228686
E: A.J.Overend@lboro.ac.uk 

Notes for editors:

1.   The study ‘Experts judgements of management journal quality: the role of identity concerns in a contested domain’ by Kim .O Peters, Kevin Daniels, Gerard P. Hodgkinson and S. Alexander Haslam’ was presented at the prestigious 2009 Academy of Management Meeting, Chicago, IL in August.

2.   Loughborough is one of the country’s leading universities, with an international reputation for research that matters, excellence in teaching, strong links with industry, and unrivalled sporting achievement and its underpinning academic disciplines.

It was awarded the coveted Sunday Times University of the Year 2008-09 title, and is consistently ranked in the top twenty of UK universities in national newspaper league tables. In the 2009 National Student Survey, Loughborough was voted one of the top five universities in the UK, and has topped the Times Higher Education league for the UK’s Best Student Experience every year since the poll's inception in 2006. In recognition of its contribution to the sector, the University has been awarded six Queen's Anniversary Prizes.

Its Business School is recognised internationally by EQUIS accreditation for the excellence of its activities and is among just 5% of business schools in the world to have achieved this prestigious accreditation.

The Business School is frequently regarded as one of the top-ten business schools in the UK; its teaching is consistently rated as ‘excellent’ and its programmes are informed by world-class research, with 60% of research rated as 'world leading’ or ‘internationally excellent' by the 2008 RAE. The 2008 National Students Survey listed us at 4th in the UK for Management Studies, and joint 5th for Accounting.

Search news


Contact us

Public Relations Office
Loughborough University
Leicestershire
LE11 3TU

Tel: 01509 222224
Fax: 01509 223902
Email


Find out more


Page maintained by: publicrelations@lboro.ac.uk
This page was last updated on Wednesday September 21, 2011 10:46