Scientific Impact and Meta-Evaluation of Open and Closed Access Disqualified Sistematic Reviews Publisher



Shahrakimohammadi A ; Keikha L ; Khani F
Source: Scientometrics Research Journal Published:2025

Abstract

Purpose: Retraction of articles is highly necessary to maintain the integrity and credibility of published scientific articles and to inform all audiences about potential serious scientific and ethical issues within these articles. Studies indicate that, in the field of medicine, many articles continue to receive citations even after being retracted. Altmetrics is a method based on scientific data that evaluates the impact of research and studies on social media by analyzing interactions and contributions from researchers. This study aims to investigate the scientific impact and Altmetrics of retracted systematic reviews with open and closed access. Methodology: PubMed, Scopus, and Web of Science databases were searched using appropriate keywords. A total of 433 articles were initially retrieved; 143 duplicates were removed due to duplicates, and 81 articles were excluded for reasons such as being non-systematic reviews, not retracted, or unrelated to the medical field. Ultimately, 209 articles met the inclusion criteria for this study. Data were extracted on access type, reasons for retraction, Altmetric scores, and article citations. Retracted articles were independently reviewed by two researchers. All required data were recorded using a checklist designed in Excel, with the dataset updated as of June 2023. Firstly, EndNote version 9 was used to screen the data. Then, an Altmetrics and citation data extraction checklist was designed using Excel 2019. Finally, the data were analyzed using Excel 2019 and SPSS version 21 software. Descriptive statistics and Pearson's correlation coefficient test were employed for analytical statistics. Findings: The findings showed that more than 67% of the articles were open access, with an average age of 6.39 ± 3.15 years. The publication period for all reviewed articles spanned from 2001 to 2022, while openaccess articles were published between 2008 and 2022. These articles were retracted between 2011 and 2022. The most common reasons for retraction were honest errors for open-access articles and fake peer review for closed-access articles. The average Altmetrics score for all retracted articles was 42.44. The average Altmetrics score of retracted systematic review articles with open access was approximately 78.83 ± 463.56. The altmetrics analysis revealed that 64 of the reviewed articles had no Altmetrics score, 33 of which were closed-access articles. According to the Web of Science (WOS) database, the total number of citations for the retracted systematic review articles was 3,713. Open-access articles received more citations than closed-access articles, with an average of 17.78 ± 45.99 citations. Most of the Twitter users sharing the reviewed systematic review/meta-analysis (SR/MA) articles were members of the public, regardless of access type. It should be noted that some articles were mentioned by multiple readers on social networks. Detailed Mendeley access data, including country names, was available for only 28% of the retracted SR/MA articles. Most readers of retracted OA and CA articles on two social networks, Twitter and Mendeley, were from the United States. The results of the Pearson correlation test showed a significant correlation between the Altmetrics score and the total number of citations of the reviewed articles in the Scopus and Web of Science (WOS) databases (p-value = 0.000). Conclusion: Providing timely information to the audience and using new technologies can lead to a reduction in the formal and informal use of these types of articles. Our findings indicate that retracted articles may continue to be used within both scientific and social communities even after their retraction. Specifically, retracted systematic review studies— often withdrawn due to scientific errors in their results and methods— may still have parts of their findings disseminated through social networks. This practice can influence clinical decision-making and potentially endanger public health. Since many researchers share their scientific work on academic social networks such as ResearchGate and Academia.edu, as well as on public platforms like X, Facebook, and YouTube, these networks should incorporate intelligent software designed to analyze and evaluate the validity of the scientific studies and sources referenced within their content. This issue requires informing senior social network managers about the need for increased monitoring of published scientific content. Providing timely information to the audience, along with utilizing new technologies, will help reduce the dissemination of such articles. Additionally, the future application of artificial intelligence techniques may minimize the number of citations to retracted articles, especially reputable articles. © 2025 Elsevier B.V., All rights reserved.