A worth noting initiative for the CASPER Project is the San Francisco Declaration on Research Assessment (DORA).
DORA was developed during the 2012 Annual Meeting of the American Society for Cell Biology in San Francisco. But today, it is a global initiative that covers all scholarly disciplines and key stakeholders including funders, publishers, professional societies, institutions, and researchers. More than 1.900 organisations and 15.500 individuals have signed it and it is available in 21 languages, which proves its international relevance. As CASPER aims at examining the feasibility of establishing an award or certification scheme for gender equality in research organisations at the European Union level, it is useful to explore and establish links with DORA.
The Declaration starts by recognising the need for improving how the output of academic research is evaluated by publishers, academic institutions, funders and other parties. It also acknowledges that to change this practice requires the engagement of all the stakeholders in the academic field. Therefore, DORA provides one general recommendation and then targeted ones for each type of institution.
As for academic audiences in general, DORA recommends not using “journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions” (Recommendation 1). In what concerns funding agencies, DORA encourages them to be clear regarding the criteria used at the time of evaluating the scientific productivity of a grant applicant –especially those in early stages– (Recommendation 2) and to consider all the research outputs (including databases and software) and not only publications (Recommendation 3).
For research performing organisations (RPOs), DORA emphasises the importance of establishing beforehand the recruitment and career advancement indicators that will be considered and to prioritise quality over quantity (Recommendation 4). Moreover, it calls upon RPOs to pay attention as well to the impact that research has on policymaking and hands-on practice (Recommendation 5).
Several suggestions are put in place for publishers, from which we would like to highlight number 8, which encourages “responsible authorship practices and the provision of information about the specific contributions of each author”. While this is not addressed in the Declaration, a proper quotation system that allows the readers to see the full names of the authors is key from a gender equality perspective too. Creators of metrics and indexes were not left out either and they are required to provide open and transparent data and mechanisms (Recommendation 11) and to have a zero-tolerance policy towards data manipulation.
Researchers, when they take part in recruiting and selection committees, are asked to take into consideration the content of a work rather than the metrics (Recommendation 15); to rely on and cite primary literature instead of reviews (Recommendation 16) and to “challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs” (Recommendation 18).
As stated in “Room for everyone’s talent” (https://www.vsnu.nl/recognitionandrewards/wp-content/uploads/2019/11/Pos...) “the implicit and overly one-sided emphasis on traditional, quantifiable output indicators (e.g., number of publications, h-index and journal impact factor) is one of the causes of heavy workload and it can also upset the balance between academic fields”. Since we see great potential in DORA, we invite all our readers to check out its website (https://sfdora.org/) and to further reflect how these recommendations can be included in your organisations to make research evaluation more equal, gender-sensitive, diverse but most importantly, fairer.
*Photo by Vitaly Taranov from Unsplash.