Part of KE's work on Open Scholarship aims to enhance the evaluation of research and researchers. This currently does not cover recognition of non-academic contributions to make Open Scholarship work, such as activities to open up and curate data for re-use, or making research results findable and available. Our approach is to raise more awareness on the lack of recognition in current evaluation practice and work towards a possible solution, through the development of an 'Openness Profile'.
The KE Open Scholarship Research Evaluation task & finish group works on the awareness issue, listing all academic and non-academic contributions that are essential to Open Scholarship and should be recognised when evaluating research. The group also works on the Openness Profile, a tool that is meant to allow evaluation of currently ignored contributions that are essential for Open Scholarship. For the development of the Openness Profile we seek involvement of various key stakeholders and alignment with current identifiers such as DOI and ORCID iD.
By demonstrating the immaturity of current research evaluation practice, and by developing the Openness Profile tool, KE supports researchers as well as non-researchers to get credit for all their contributions that make Open Scholarship possible. Our ambition is that recognition of these essential activities becomes part of standard research evaluation routine.
Interim report - Openness Profile: Defining the Concepts
We have documented our ongoing investigation into the need for and value of new evaluation approaches of people conducting open scholarship and their outputs, in the newly published report 'Openness Profile: Defining the Concepts'.
The report provides an extensive overview of strategies, barriers, and community needs regarding openness and explores what contributions an Openness Profile, as introduced in this report, can make to enable desired openness and fairer assessment in research.
It is based on interviews conducted with 19 research contributors, both research conductors as well as research supporting professionals. The focus areas of the interviews were strategies, mandates, skills, community norms, appraisals/evaluations, non-individual profiles, barriers, incentives, and feedback on the OP concept itself.
The study showed that:
- The OP could serve the purposes of being part of their annual review, informing decision making or creating incentives/metrics at their organization.
- There is a frustration with current incentive structures and cultural inertia is very common, which translated into a desire for systemic change in how contributions to scholarship are valued and who is credited.
- It was frequently found that openness is either not currently discussed in detail in interviewees' personal evaluations, or that interviewees had only informal evaluations or none at all.
You can read the full report here.
A stakeholder workshop was planned for April 2020, but postponed due to COVID-19 restrictions. The face-to-face workshop was replaced by an online workshop held in early July 2020, in which key experts in open scholarship discussed and challenged the Openness Profile concept and advised on possible next steps. Between August-September 2020, focus groups of various stakeholder groups such as funders, HEIs and other open evaluation projects, debated and analysed the potential benefit of an operational Openness Profile infrastructure to achieve their ambitions.
Publication of our final report ‘Knowledge Exchange Openness Profile: A reference model for the evaluation of open scholarship' is expected in February 2021.
Members of the KE Open Scholarship Research Evaluation task & finish group
- Clifford Tatum (lead expert) - CWTS, Leiden University
- Heidi Laine - CSC - IT Center for Science
- Verena Weigert - Jisc
- Frédéric Hélein - Institut de Mathématiques de Jussieu-Paris Rive Gauche
- Rachel Bruce - Jisc
- Lorna Wildgaard - The Royal Danish Library (ORCID ID orcid.org/0000-0002-3900-5058)
- Daniel Beucke - University of Goettingen
- Joonas Nikkanen - CSC - IT Center for Science
- Serge Bauin - The French National Centre for Scientific Research (CNRS)
- Josefine Nordling (KE lead) - CSC - IT Center for Science
- Jean-Francois Nominé (KE co-lead) - The French National Centre for Scientific Research (CNRS)
Consultants working on this KE activity: