Humboldt-Universität zu Berlin - Startseite

Proliferation of national research evaluation procedures is arguably one of the main factors shaping the contemporary institutional landscape of science in Europe. Consequently, we observe a growing interest in research evaluation’s impact on the behaviour of both researchers and research institutions. So far, however, we have limited understanding of a social process behind the construction of research evaluation procedures themselves. Hidden behind closed doors of policy-forming chambers and committees, they often remain beyond researchers’ view. In effect, research evaluation forms a black box, which can be studied primarily through its inputs and outputs. In an attempt to open this black box, this study follows in the direction set by the sociology of quantification and asks: what shapes the production of numbers in the context of research evaluation? It posits this question in the context of the Polish science system, as it takes advantage of its unique character, i.e. high participation of the academic community in the creation of evaluation procedures. In particular, this study draws from the recent experience of creating the Polish Journal Ranking, a crucial policy instrument of  research evaluation system in Poland, which is studied in reference to two disciplines, biology and history. A mixed method approach is adopted, combining semi-structured interviews with actors of this process with a quantitative analysis of consecutive changes in the ranking on the different stages of its construction. This, in turn, sheds light on a complex social process of creating evaluative homogeneity out of disciplinary heterogeneity. The study further argues that this process is dictated by how actors position themselves in relation to two opposing forces: those favouring the homogenization of research and those maintaining its heterogeneity. As these forces remain imbalanced, the study concludes with a call for further exploration of the interplay between the forces of homogenization and heterogenization and how the tensions between them are mediated within a performance-based research funding system.

Notes on contributors:

Emanuel Kulczycki – the head of Scholarly Communication Research Group (Adam Mickiewicz University in Poznań) and a policy advisor to the Ministry of Education and Science in Poland. In 2018–2020, the chair of the European Network for Research Evaluation in the Social Sciences and the Humanities (ENRESSH). His recent papers were published in ‘Nature’, ‘Research Evaluation’, ‘Journal of Informetrics’, and ‘Scientometrics’. More information: emanuelkulczycki.com

Krystian Szadkowski – researcher at the Scholarly Communication Research Group of Adam Mickiewicz University. His interests cover Marxist political economy and transformations of higher education systems in Central Eastern Europe, as well as the issues of the public and the common in higher education. He worked as a researcher for Education International and as a consultant in policy projects funded by Polish Ministry of Science and Higher Education. In 2018 he was a visiting researcher at Centre for Global Higher Education, Institute of Education, University College London.

Jakub Krzeski – assistant professor at the Department of Philosophy and Social Sciences of Nicolaus Copernicus University and researcher at the Scholarly Communication Research Group. He received his degree in 2021 defending his dissertation “A philosophical account of metrological conflict in the field of science evaluation” at the Faculty of Philosophy, Adam Mickiewicz University. His research interests focus on the social theory of quantification, critical theory and political ontology. He is currently carrying out a research project entitled “Struggles over measures: The case study of research evaluation in biology in Poland” funded by the National Science Centre.