European Commission publishes new recommendations for research assessment

Two reports propose reforms in evaluation to include criteria that take into account Open Science paradigms

LC
Lorena Caliman
02 december, 2021≈ 7 min read

Two reports recently published by the European Commission discuss the importance of reviewing the research assessment models currently in place, broadening the qualitative focus of the evaluations and taking into account the paradigms of Open Science. The first report, published on 26 November, is a scoping document summarising the results of extensive consultations with European and international partners that took place from March to November this year. The second report, published on Monday 29th, presents recommendations arising from a workshop with beneficiaries of the Science with and for Society (SwafS) project portfolio in the area of Responsible Research and Innovation.

Towards a reform of the research assessment system

The scoping report "Towards a reform of the research assessment system" encompassed a consultation that identified objectives and lines for a reform of the research assessment system in the European Research Area and also globally. The idea of the proposal is to discuss how to facilitate and accelerate this reform so that the quality, performance and impact of research and researchers are assessed on the basis of more appropriate processes and criteria.

It is proposed that the reform should follow principles and actions to be agreed among the organisations that have responsibility for defining criteria and processes for evaluating projects, researchers, units and research institutions. The implementation of these principles would consist of the signing of a European agreement by research funding organisations, research organisations, national and regional authorities and agencies, as well as their associations.

The starting point for the report was the European Commission's 2018 recommendation to member states to define and implement clear policies to reward a culture of collaboration and sharing of knowledge and data. The current evaluation system is considered limited, as it uses inadequate and too narrow methods to assess the quality, performance and impact of research and researchers. The aim of the reform is for research and researchers to be evaluated on their intrinsic merits and performance rather than only on the number of publications and on where they are published, promoting a more qualitative peer-reviewed judgement, supported by a more responsible use of quantitative indicators.

The text highlights, among other points, that "the race for publications – the so-called publish-or-perish culture – comes at the expense of quality, integrity, and trust in research. Also, using the JIF [Journal Impact Factor] as a proxy for quality of research is shown to be inappropriate". The report, therefore, argues that a reformed assessment system should be based on qualitative judgement and that it should "acknowledge the full range of research outputs and processes, should reflect the diversity of research-related activities such as mentoring, leadership roles or outreach and interaction with society, and should take account of the diversity of individual career paths".

Also research units should, according to the European Commission publication, be assessed "not only on the basis of their research outputs but also on their relative contribution to research missions, while research assessment by research funders should acknowledge multi-, inter-, and trans-disciplinary research as well as research contributing to innovation and societal impact".

The European Commission's proposal also foresees that the agreement should be accompanied by supporting measures, such as financial support by governments and funding institutions at European and national level, awareness campaigns for researchers, the establishment of alliances of European universities, the updating of the European researchers' charter and code of conduct for the recruitment of researchers, and an approach to dialogue with national and regional authorities with the intention of reducing obstacles and legal barriers to change. Finally, the Commission further recommends international dialogues to advance alignment at the global level.

Considering the different profiles, the report also highlights that reforms in the system should be appropriate for each type of assessment - research projects, researchers, research units, and research institutions. An implementation plan should be established by the signatories of the proposed agreement, including deliverables, milestones and timeframes, in order to translate commitments into effective changes.

Institutional logics and Open Science in universities

The European Commission's second report, "Enabling open science and societal engagement in research", published this Monday 29th, reflects the results of the discussion at a workshop held in July. The participants, from the Science with and for Society (SwafS) programme, discussed how open science and societal engagement could become the norm for research organisations across the European research area, with a particular focus on universities. The paper shows the assessment of the alignment of Open Science principles with three institutional logics that shape practices in European universities: the logic of the ivory tower, the logic of the utilitarian university, and the logic of managed bureaucracy.

From these assessments, the participants outlined recommendations for reforms to be made in European universities, among which the first highlight is the need for universities and other research organisations to reform the criteria, metrics and processes that support recruitment and career progression so that Open Science practices can be rewarded.

The recommendations include as well: the institutionalization of Open Science in universities and organizations as a long term project; the continuity of the reforms in indicators, measures and processes used in the evaluations of programs, researchers and research units, in order to include the evaluation of open science practices; the assessment on the involvement of civil society organizations and citizens in applied research and innovation projects and programs, among others.

As highlighted in the documents, a European initiative dedicated to improving research evaluation can be seen as an opportunity for universities to consider how to best balance the various activities of academics in their evaluation. The reports affirm the need to reward open scientific practices in terms of open collaboration and the early sharing of knowledge and data that can lead to higher quality, efficiency, impact and trust in research.

| More information |

Read the reports:

Towards a reform of the research assessment system

Enabling open science and societal engagement in research