The Role of Open Science in the Evaluation of Research Work

There are not yet any generally applicable standards for the integration of Open Science in the evaluation of research work. The measurement of the success of scientific work that has applied until now, primarily on the basis of publications (citations), does not reflect the efforts involved and benefits to be gained from Open Science. For open science practices to be worthwhile for scholarly reputation, there would need to be new methods of evaluating achievement that reward Open Science practices. This can take place if they also include the publication of research data and methods, review activities or comments.

Attempts are already being made to develop supplementary indicators that supplement the citation-based indicators, such as the “H Index” and “Journal Impact Factor” to include information on the occurrence of scientific results outside specialist journals. These attempts place greater emphasis upon aspects of digital science. For instance, altmetrics measure resonance in social media. However, altmetrics cannot be unreservedly recommended for the measurement of Open Science, because, like all other quantitative indicators, they can give preference to quantitative values (such as citations and retweets) over qualities which cannot be counted (such as the quality of the research).

A declaration that is of central importance for the Open Science movement, the Declaration on Research Assessment (DORA), which has been signed by numerous stakeholders in the field of research, calls for no journal-based metrics such as the journal impact factor, to be used as a replacement for the evaluation of the quality of an individual research article. Instead, new standards must be created for the evaluation of the contributions of individual researchers or for decisions about their appointment, promotion or funding, which support making scientific results accessible without restrictions. Furthermore, research institutions and research funding institutions should also consider, alongside publications, all other research achievements (including datasets and software) as well as a series of other factors such as qualitative indicators for the effects of the research, for example influence on politics. As this topic is very complex, scientific communities must decide for themselves how they want to implement it.