The impact of classification systems in the evaluation of the research performance of the Leiden Ranking universities Articles uri icon

publication date

  • February 2018

start page

  • 1046

end page

  • 1053

issue

  • 8

volume

  • 69

International Standard Serial Number (ISSN)

  • 2330-1635

Electronic International Standard Serial Number (EISSN)

  • 2330-1643

abstract

  • In this article, we investigate the consequences of choosing different classification systems—namely, the way publications (or journals) are assigned to scientific fields—for the ranking of research units. We study the impact of this choice on the ranking of 500 universities in the 2013 edition of the Leiden Ranking in two cases. First, we compare a Web of Science (WoS) journal-level classification system, consisting of 236 subject categories, and a publication-level algorithmically constructed system, denoted G8, consisting of 5,119 clusters. The result is that the consequences of the move from the WoS to the G8 system using the Top 1% citation impact indicator are much greater than the consequences of this move using the Top 10% indicator. Second, we compare the G8 classification system and a publication-level alternative of the same family, the G6 system, consisting of 1,363 clusters. The result is that, although less important than in the previous case, the consequences of the move from the G6 to the G8 system under the Top 1% indicator are still of a large order of magnitude.