Open Science and Alternative Metrics in Research Evaluation

The current measurement of scientific success, based primarily on citations, reflects neither the effort nor the benefits of Open Science. New methods of performance evaluation are needed that also reward practices such as the publication of research data and methods, review activities, or comments. However, universally accepted standards for this have yet to be established.

To complement citation-based indicators such as the "H Index" and the "Journal Impact Factor", alternative metrics (altmetrics) have been developed that capture how research outputs are received beyond specialist journals. Tools such as Altmetric.com or PlumX track online attention, including mentions in news media, policy documents, blogs, and social media. They may thus provide insights into the broader societal impact of research. Many publishers, including PLOS ONE, Elsevier, Springer Nature, and Wiley, also integrate such article-level metrics directly on their platforms, making alternative indicators accessible without the need for dedicated external tools.

However, altmetrics are not without limitations. Coverage and results can vary depending on the provider used, which makes scores from different tools difficult to compare directly. Most widely used services are commercially operated, which raises questions about data transparency and long-term availability. In economics and business studies, coverage is still relatively sparse, partly because much research circulates as working papers on repositories such as SSRN or RePEC, which most altmetric tools do not systematically track. Download and access statistics from these repositories can nonetheless serve as a practical, discipline-specific proxy for measuring reach.

Of central importance to the Open Science movement is the Declaration on Research Assessment (DORA), which has been signed by numerous stakeholders in the research community. It calls for journal-based metrics such as the Journal Impact Factor not to be used as a substitute for assessing the quality of individual research. Instead, new standards must be created for evaluating researchers' contributions for instance regarding appointment, promotion, or funding that go beyond publication-based metrics and support open access to scientific results. Research institutions and funding bodies should consider not only publications but all research outputs (including datasets and software) alongside qualitative indicators of impact, such as influence on policy. As this is a highly complex issue, scientific communities themselves must decide how to implement it.

For those wishing to deepen their understanding of these principles, DORA also offers a free, self-paced Introductory Course to Responsible Research Assessment.