Responsible use of metrics
Durham University is developing its approach as to how and where publication metrics are used as research indicators at an institutional level, and recognises the importance of using bibliometrics responsibly.
In July 2015, The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management outlined five keys areas to inform the responsible use of metrics:
- Robustness: basing metrics on the best possible data in terms of accuracy and scope;
- Humility: recognising that quantitative evaluation should support – but not supplant – qualitative, expert assessment;
- Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results;
- Diversity: accounting for variation by field, and using a range of indicators to reflect and support a plurality of research and researcher career paths across the system;
- Reflexivity: recognising and anticipating the systemic and potential effects of indicators, and updating them in response.
The report made a number of recommendations to UK HEIs, including to;
"carefully select quantitative indicators that are appropriate to their institutional aims and context ... explain why they are using these as a means to achieve particular ends ... [and] where possible, alternative indicators that support equality and diversity should be identified and included."
... to provide
"Clear communication of the rationale for selecting particular indicators, and how they will be used as a management tool... [and] consider signing up to DORA, or drawing on its principles and tailoring them to their institutional contexts."
... and to
"pay due attention to the equality and diversity implications of research assessment choices"
Following discussions at University Research Committee on how to implement the recommendations of the Metric Tide report, work was undertaken to consult stakeholders and take into account the needs of researchers and authors, as well as consideration of existing external developments such as the San Francisco Declaration on Research Assessment (DORA), the Leiden Manifesto and the development and use of Snowball Metrics.
The outcomes of this work will be made available via this page once concluded.
- The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management
- San Francisco Declaration on Research Assessment (DORA)
- The Leiden Manifesto for Research Metrics
- UUK Forum for Responsible Metrics
- Snowball Metrics: Standardised metrics for the sector, by the sector
What is DORA?
Some of the key themes of DORA include:
- To eliminate the use of journal-based metrics, such as the Journal Impact Factor, in considerations for funding, appointment and promotion.
- To assess research on its own merits, not the venue of publication.
- To realise opportunities of online publication, such as the relaxation of limits on the number of references in published articles.
Durham University and DORA
On 16th October 2018, University Senate approved the proposal that the University became a signatory to the San Francisco Declaration on Research Assessment (DoRA), whilst "noting the importance of monographs in particular academic disciplines."
On 8th November 2018, Professor Claire Warwick, Pro-Vice-Chancellor (Research) signed DoRA on behalf of Durham University.
Who else has signed DORA?
Other UK organisational signatories include:
|Research England (as HEFCE)||UK Research Councils||Wellcome Trust|
|Imperial College London||King's College London||Oxford University|
|University College London||University of Bristol||University of Liverpool|
|University of Manchester||University of York||Newcastle University|
Following discussion at University Research Committee, a University statement of principles, and a University policy on responsible metrics were opened up to a University-wide consultation which closed on September 21st 2018:
- Both the policy and statement were agreed at University Executive Committee on 11th September 2018, subject to an amendment to reflect points raised on achieving research excellence, publishing in high quality publications and a clearer statement on the transparent use of metrics by the University.
- The amended policy and statement were subsequently agreed by University Senate on 16th October 2018.
The agreed documents can be found at the links below:
- [FINAL] University Statement on the Responsible Use of Metrics
- [FINAL] University Policy on Responsible Metrics