The UOC aligns with new international trends in research assessment

Universities are facing the challenge of increasing responsibilities in the assessment of their research and their staff members' academic careers. Institutions such as the Universitat Oberta de Catalunya (UOC) are looking for a fairer and more responsible way to assess research based on a wide and diverse range of criteria and merits. In this spirit, last year the university joined the international Coalition for Advancing Research Assessment (CoARA), a European movement seeking to improve research evaluation. In addition, last semester it asked its teaching and research staff to help it assess its capabilities and internal preparation for adoption of the CoARA principles in its research assessment processes and to find out about its research staff's opinions and concerns before starting to design specific actions. The starting point was 2019, when the university approved its Open Knowledge Action Plan and signed the San Francisco Declaration (DORA), an international movement to promote quality-based research assessment.

According to Julie Wilson (associate dean for research at the Faculty of Economics and Business and researcher of the New Perspectives in Tourism and Leisure research group, NOUTUR), who is leading, together with Ciro Llueca (Deputy General Manager for Research and Knowledge Transfer) a group working on a reform plan for advancing research assessment, involving the research staff "has served to examine the need to modify our own research assessment practices. The questionnaire they were sent places importance on recognizing the diversity of research achievements – such as merits, profiles, tasks, careers and subject areas – as well as making a more responsible use of quantitative metrics to provide more context wherever and whenever necessary."

The analysis, which involved close to one hundred researchers, "illustrates our training and ability as an institution to assess research requests, projects and processes according to the principles of DORA and CoARA, as well, of course, as assessing teaching staff", said Julie Wilson. In fact, more than half the participating researchers already have some experience with broader methodologies – which take into account the diversity of research activities and their practices – although there are also many training needs and requests in this regard. The analysis also shows that we need, in some cases, to know more about current assessment processes and, in others, to promote greater transparency in relation to the criteria of some internal calls for proposals.

The UOC's questionnaire for teaching and research staff on the state of research assessment shows that the changes to be applied will require specific training and skills-acquisition to reduce possible conflicts of interest, including bringing in external assessors to reinforce the critical mass at the university.

The analysis also shows that there was a 'before' and an 'after' the signing of the CoARA agreement in December 2022 in terms of the UOC staff's perceptions of what the rest of the system would do. The teaching and research staff didn't know what university quality agencies – such as the National Agency for Quality Assessment and Accreditation of Spain (ANECA) and the Catalan University Quality Assurance Agency (AQU) – and other organizations such as the Conference of Rectors of Spanish Universities (CRUE) would do. According to the people responsible for the analysis, the organization of the Spanish chapter in CoARA, with the participation of most universities and the Spanish and Catalan quality agencies, among others, as well as the steps taken by the assessment agencies, help to understand that "changes are being considered throughout the system, not just individually".

How research assessment at the UOC has changed

In the preliminary work carried out for the research assessment analysis with the UOC's teaching and research staff, Llorenç Arguimbau, an expert in this field, was asked to analyse research assessment in the UOC's internal calls for proposals since signing DORA in 2019. According to the study presented, which analysed the period 2019-2022, "regarding the methodology for assessing candidates and their projects, an analysis of the terms of the calls for proposals shows that the UOC has evolved towards a more qualitative assessment" and, as required by DORA, "automatic scales based on the number of articles published and the impact factor of the journals involved have been removed from the most recent calls' selection criteria".

Arguimbau also noted that the assessment criteria for internal calls for proposals have evolved significantly in aspects such as scientific quality (applications are currently rated based on aspects such as the scientific and technical quality of their contents), publication indicators (alternative indicators and citations can now be submitted), social impact (responses to global challenges need to be explored), context, diversity of results, transparency and openness (the question of whether results are open access is also asked).

The UOC's CoARA Plan 2024-2027

The UOC's participation in CoARA includes drawing up a four-year action plan in which the university must consider how to address the 10 commitments of the agreement that was signed a year ago. The UOC's CoARA Plan 2024-2027 – sponsored by Xavier Vilajosana, Vice Rector for Research, Knowledge Transfer and Entrepreneurship and lead researcher of the Wireless Networks (WINE) group of the Internet Interdisciplinary Institute (IN3) – will be presented before the end of the year following a collaborative process for its final validation.

The group in charge of drafting the Plan started last summer. It is being coordinated by Pastora Martínez-Samper, the university's commissioner for international action, and the participants include the aforementioned Julie Wilson, Víctor García Font, member of the Faculty of Computer Science, Multimedia and Telecommunications and researcher in the IN3's K-ryptography and Information Security for Open Networks (KISON) group; Israel Rodríguez, a researcher at the IN3 and leader of the Care and Preparedness in the Network Society (CareNet) group; and Bego Aguilera, director of open science at the UOC's Office of the Deputy General Manager for Research and Knowledge Transfer. The working group plans to involve the academic community in the autumn, in line with the UOC's commitment to promoting debate and consensus, working collectively to achieve the changes in research assessment needed by the academic community.

Participation in CoARA and organizations working in this field

The UOC takes part in several national and international forums that address the challenges of research assessment. Xavier Vilajosana, Vice Rector for Research, Knowledge Transfer and Entrepreneurship, represents the UOC in the Spain National Chapter of CoARA, promoted by ANECA, CRUE and CSIC (the Spanish National Research Council). In addition, Pastora Martínez-Samper was appointed member of the Commission for the evaluation and monitoring of the state accreditation system of ANECA and is one of two coordinators of the Reforming Academic Career Assessment working group selected by CoARA, which is coordinated by the European University Association (EUA) and has more than 30 academic organizations participating from all over Europe.

UOC R&I

The UOC's research and innovation (R&I) is helping overcome pressing challenges faced by global societies in the 21st century by studying interactions between technology and human & social sciences with a specific focus on the network society, e-learning and e-health.

Over 500 researchers and more than 50 research groups work in the UOC's seven faculties, its eLearning Research programme and its two research centres: the Internet Interdisciplinary Institute (IN3) and the eHealth Center (eHC).

The university also develops online learning innovations at its eLearning Innovation Center (eLinC), as well as UOC community entrepreneurship and knowledge transfer via the Hubbik platform.

Open knowledge and the goals of the United Nations 2030 Agenda for Sustainable Development serve as strategic pillars for the UOC's teaching, research and innovation. More information: research.uoc.edu.