Responsible use of metrics
Durham University is developing its approach as to how and where publication metrics are used as research indicators at an institutional level, and recognises the importance of using bibliometrics responsibly.
Timetable of consultation
The University has reviewed how best to implement to the recommendations of the Metric Tide Report (2015):
- June 2016: Metric Tide Recommendations and use of metrics was discussed at University Research Committee [Minutes: Item 29].
- recommendation: further internal consultation required
- recommendation: consultation should be followed by clear policy and guidance.
- suggestion: 'Future Leaders' programme should be utilised to form a cross-university working group to consult with stakeholders and recommend direction of travel.
- Mar-Nov 2017: Future Leader's Group (Academic and Professional Support staff membership) consultation with stakeholders across the University.
- work was undertaken to consult stakeholders and take into account the needs of researchers and authors, as well as consideration of existing external developments such as the San Francisco Declaration on Research Assessment (DORA), the Leiden Manifesto and the development and use of Snowball Metrics.
- November 2017: Presentation to members of University Executive Committee
- January 2018: Future Leader's presented recommendations from stakeholder consultation to University Research Management Committee [Minutes: Item 27]
- recommendation: University required a clear public statement of principles and a policy on the responsible use of metrics.
- recommendation: University Responsible Metrics Working Group should be established to implement recommendations.
- June 2018: Working Group reported recommendations and draft public statement and policy to University Research Committee [Minutes: Item 47]
- recommendation: Policy and Statement (subject to minor amendment) to be put out for consultation.
- recommendation: If consultation successful, approved for escalation to UEC and Senate for approval.
- Jul-Sep 2018: Policy and Statement opened to consulation.
- 11 September 2018: Approved by University Executive Committee subject to an amendment to reflect points raised on achieving research excellence, publishing in high quality publications and a clearer statement on the transparent use of metrics by the University. [Minutes: Item 24]
- 16 October 2018: Approved by University Senate [Minutes: Item 14]
- The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management
- Research England: Research Metrics Review
- San Francisco Declaration on Research Assessment (DORA)
- The Leiden Manifesto for Research Metrics
- UUK Forum for Responsible Metrics
- Snowball Metrics: Standardised metrics for the sector, by the sector
In July 2015, The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management outlined five keys areas to inform the responsible use of metrics:
- Robustness: basing metrics on the best possible data in terms of accuracy and scope;
- Humility: recognising that quantitative evaluation should support – but not supplant – qualitative, expert assessment;
- Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results;
- Diversity: accounting for variation by field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system;
- Reflexivity: recognising and anticipating the systemic and potential effects of indicators, and updating them in response.
The report made a number of recommendations to UK HEIs, including to;
"carefully select quantitative indicators that are appropriate to their institutional aims and context ... explain why they are using these as a means to achieve particular ends ... [and] where possible, alternative indicators that support equality and diversity should be identified and included."
... to provide
"Clear communication of the rationale for selecting particular indicators, and how they will be used as a management tool... [and] consider signing up to DORA, or drawing on its principles and tailoring them to their institutional contexts."
... and to
"pay due attention to the equality and diversity implications of research assessment choices"
What is DORA?
Some of the key themes of DORA include:
- To eliminate the use of journal-based metrics, such as the Journal Impact Factor, in considerations for funding, appointment and promotion.
- To assess research on its own merits, not the venue of publication.
- To realise opportunities of online publication, such as the relaxation of limits on the number of references in published articles.
Durham University and DORA
On 16th October 2018, University Senate approved the proposal that the University became a signatory to the San Francisco Declaration on Research Assessment (DoRA), whilst "noting the importance of monographs in particular academic disciplines."
On 8th November 2018, Professor Claire Warwick, Pro-Vice-Chancellor (Research) signed DoRA on behalf of Durham University.
Who else has signed DORA?
Other UK organisational signatories include:
|Research England (as HEFCE)||UK Research Councils||Wellcome Trust|
|Imperial College London||King's College London||Oxford University|
|University College London||University of Bristol||University of Liverpool|
|University of Manchester||University of York||Newcastle University|
Following discussion at University Research Committee, a University statement of principles, and a University policy on responsible metrics were opened up to a University-wide consultation which closed on September 21st 2018:
- Both the policy and statement were agreed at University Executive Committee on 11th September 2018, subject to an amendment to reflect points raised on achieving research excellence, publishing in high quality publications and a clearer statement on the transparent use of metrics by the University.
- The amended policy and statement were subsequently agreed by University Senate on 16th October 2018.
The agreed documents can be found at the links below:
- [FINAL] University Statement on the Responsible Use of Metrics
- [FINAL] University Policy on Responsible Metrics
Your Academic Liaison Librarian
Metrics Quick Links
Metrics Top Tips
- Always use quantitative metrics together with qualitative inputs, such as expert opinion or peer review.
- Always use more than one quantitative metric to get the richest perspective.
- If comparing entities, normalise the data to account for differences in subject area, year of publication and document type.
See our pages on Responsible Metrics for further information.