Cookies

We use cookies to ensure that we give you the best experience on our website. You can change your cookie settings at any time. Otherwise, we'll assume you're OK to continue.

University Library

Bibliometrics: Journal, article and author metrics


Bibliometrics

Bibliometrics is the process of extracting measurable data through the statistical analysis of texts, and information about how the texts are being used. For researchers this might be:

  • to determine how many times a researcher's work has been cited in key literaturE;
  • (SCHOLARLY IMPACT) to explore the impact of their field, the impact of a set of researchers, or the impact of a particular paper
  • (EVALUATION) a means of measuring patterns of authorship, publication, and the use of literature to aid in evaluation and review
  • (COLLABORATION) to help identify if an author is attracting citations from outside their main field of study, and thus highlight potential opportunities for cross-disciplinary collaboration.
  • (DISSEMINATION) to help guide or influence publication strategies (eg deciding where to publish research in to obtain maximum visibility)

Bibliometric tools, such as citation analysis, remain controversial as proxy indicators of the quality of published research. For example, citations do not measure the extent or particiaption in multi-authored publications, and negative citations will contribute as positive indicators of impact.

Traditional bibliometric tools can also potentially underestimate or completely fail to highlight the true impact of a given piece of research as they tend to rely on data sets restricted to academic publications and citation indexes and lack a measure of the 'immediacy' of any initial impact which may not necessarily translate into published citing articles.

Alternative Metrics (or Altmetrics) are developing branch of metrics which look at non-scholarly as well as scholarly sources such as social media and non-academic online sources.

This page looks at some of the tools, resources and training available for you as a researcher in exploring the various journal, article and author metrics you come across or wish to use.


Citations, citation counts and citation indexes

Citations

Citations are a reference to a source or underpinning set of data for the purpose of acknowledging their relevance to the topic of discussion. Different publication cultures in different disciplines mean that in some subject areas you are more likely to be expected to cite extensively the work that your scholarship and research is building upon, than in others.

Citation counting

The number of citations an article receives is one indicator of the "academic impact" of the article, based upon its popularity in terms of how many people have read and then refered to that research. A high citation count is not an indicator of high quality. Indeed, an article may be widely cited as an example of bad practice. It should also be noted that there are wildly varying traditions of citiation in different disciplines and therefore what is highly cited in one field may only represent an average citation count in another. There are various tools available which enable you to track academic debate based on who has cited a given work. This might be useful for:

  • Tracking the impact of your own research (for means of engaging with other researchers, identifying opportunities for collaboration or simply identifying more fruitful publictaion within which to publish;
  • Keeping up to date with how a particular piece of research you are interested in is received and treated by others.

Citation indexes

In order to monitor citations, you first of all of course need a dataset recording citations across a broad enough range of publications to make any collection, counting and analysis in any way meaningful.

Example: JSTOR provides an indication of the number of times an article has been cited, but only counts those citations in journals which also appear on the JSTOR platform. For many subject areas, this might only be a very small proportion of the full range of academic titles where citations might have appeared.

There are three main datasets available which each have multi-disciplinary coverage of available research and the citations associated with that research.

  • Web of Science and related products (provided by Thomson Reuters)
  • Scopus and related producst (provided by Elsevier) [Not available at Durham University]
  • Google Scholar
These services allow you to:
  • count citations - find out how many others have cited a given work
  • cited reference search - find out which articles have cited a previously published work
  • citation alerts - find out when a newly published article cites a previously published work
  • citation reports - for a variety of statistics relating to an author
  • citation maps -find related material and shared citations [Web of Science]
For further information, see training available.

Journal Citation Reports® and Journal Impact Factors

Journal Citation Reports®

Every year, Thomson Reuters publishes the Journal Citation Reports® which are available via Web of Science for Science and Social Science journals indexed within the service. They give an indication of the "scholarly impact" of a particular journal using quantitive analysis of citations for articles published in that journal. They should always be considered in conjunction with advice from peers, and it is not possible to do a comparison of journals across discipline boundaries where citation cultures might vary considerably.

Journal Impact Factors (JIF)

The Journal Impact Factor (JIF) are probably the best known aspect of these reports. An impact factor represents the average number of citations in a year given to those papers in a journal that were published during the two preceding years. It is therefore a reflection of the current impact, or popularity, of a journal.

Alternative ways of measuring journal impact include the Eigenfactor and the data produced by SCImago (which utilises data from the Scopus database, rather than Web of Science).

The presentation below provides a brief overview, or for further information, see training available.



Author metrics

In addition to counting the number of citations a particular research paper has attracted or looking at the 'impact' of a particular journal in your field of research, bibliometrics can also be used to assess the output and scholarly impact of an individual author's research. These might be taken into account for:

  • applications for tenure
  • assessments for promotion or probation
  • applications for research funding awards

It is useful therefore to be aware of the various metrics that can be used, how they are generated and some of the issues around them.

  • Total number of papers / total number of citations
  • The (Hirsch) h-index of an author is the number of their publications (h) which have been cited at least h times.
  • The (Egghe) g-index of an author is a adaptation of the h-index which gives more weight to an authors more highly cited papers.

The presentation below provides a brief overview, or for further information, see training available.



Altmetrics

Traditional bibliometrics have many limitations, not the least of which is that they focus solely on citations in scholarly publications which:

  • May not be available til some time after the original article is published;
  • Only provide information on who has cited a work in a peer-reviewed article, not who else is reading, sharing, discussing or commenting outside of traditaional publications;
  • Are limited to the coverage of the particular citation data set used.

Increasingly, researchers share ideas and research at all stages of the research process online, and other researchers as well as a wider, non-academic audience, can engage with research outputs via a variety of different media such as twitter, blogs, youtube, news coverage etc.

This means that the 'impact' of you research can be seen much more quickly, and across a much broader scope, than than might be reflected using traditional bibliometric data.

Altmetrics seek to measure the impact of articles by counting mentions by social media sites and other digital communications. There are various tools available which you might be interested in exploring, such as:

As with traditional bibliometrics, there are still reasons to treat altmetrics with a level of caution (for example, it is easier to "game" altmetric data) but they can be useful to inform decisions on impact, publication strategy and potential collaboration opportunities.
We are planning to provide guidance as part of the Researcher Development Programme in 2014, alongside sessions on traditional bibliometric support.

Bibliometrics Training

Training on bibliometrics, Journal Impact Factors, author level metrics and open access are currenlty provided as part fo the Researcher Development Programme, and scheduled to run during the Michaelmas and Epiphany terms. 

A session is also delivered as part of the Leading Research Programme, aimed at early career researchers who have recently submitted, or are about to submit, their first research award application.

For any additional help or guidance, or a short one-to-one sessionyou can also contact James Bisset.


Academic Liaison Librarian for Research Support

Research Support Statement

"In a rapidly evolving research and information environment the library aims to support research in all disciplines across the research life cycle, from securing funding awards, to the dissemination and evaluation of your research and its impact.

This covers a broad range of activities, from the training and support provided through the Researcher Development and Leading Research Programmes, to enabling researchers to meet the open access and research data management requirements set by funders and the management and maintenance of our institutional repository Durham Research Online.

We are heavily involved in supporting the university's REF submissions, and can provide additional advice and services on bibliometrics and citation analysis."