Safety and efficacy of stem cell therapy: an overview protocol on published meta-analyses and evidence mapping

Introduction

Clinical and preclinical studies have identified stem cell therapy (SCT) as an emerging and promising treatment method for several diseases (1), including but not limited to age-related macular degeneration, corneal injury, diabetes mellitus, epidermolysis bullosa, knee osteoarthritis, myocardial infarction, and fire burn (1-5). However, in the era of evidence-based medicine, the basic consensus is that all healthcare decisions should be made based on the best available scientific evidence (6). A systematic review with meta-analysis (MA) is an important evidence-based evaluation method, and MA is generally used to underpin the evidence base associated with a specific question (7). MAs always play a critical role in healthcare decision-making, and can be used to inform clinical practice and future research (7,8). Nevertheless, it is noteworthy that not all MAs are useful and valued (9), and relevant problems of MAs (e.g., methodological flaws and incomplete reporting) have been pointed out in several fields (10,11). Based on our knowledge and preliminary search in PubMed, many MAs reporting efficacy and safety data concerning SCT have been identified, involving various diseases (such as chronic liver disease, diabetes mellitus, sepsis, knee osteoarthritis, stroke), in different subspecialties (3,5,12-14). For example, a recently published systematic review and MA reported that SCT is a relatively safe and effective treatment measure for diabetes mellitus (3). Zhou and colleagues investigated the efficacy and safety of SCT for chronic liver disease based on data from 24 trials and concluded SCT was also safe and effective for the treatment of chronic liver disease (12).

A systematic review of systematic reviews or MAs (i.e., overview or umbrella review) is similar to systematic review and MA (7,15), and is also a critical evidence synthesis method that can consolidate an evidence base using data from MAs, while focusing on the quality of included MAs in terms of methodological quality (i.e., rigor methodological design and implementation) and reporting quality (i.e., complete and clear reporting on key information) (7). For example, Zhu and colleagues reviewed systematic reviews and MAs concerning the use of total glucosides of paeony in the treatment of rheumatoid arthritis and found that total glucosides of paeony are an effective and safe compound for treating rheumatoid arthritis, but the methodological quality and reporting quality of included MAs were poor (11). The methodological quality and reporting quality of MAs are of paramount importance for their clinical application, because MAs with extensive methodological flaws are unreliable and misleading (7,16), and incomplete reporting usually influences the complete use of the interventions compared in MAs (17), leading to the waste of medical research and healthcare resources is accompanying (18,19). In addition, there is a lack of an evidence summary reporting the efficacy and safety data concerning SCT from MA documents. Therefore, it is necessary to perform an overview for mapping and summarizing the evidence and quality of published MAs focusing on SCT.

Here, we accordingly report a protocol of this overview concerning SCT. The objectives of the present overview include the following: (I) summarization of efficacy and safety-related evidence from published MAs focusing on SCT; (II) assessment of the methodological and reporting quality of included MAs concerning SCT; and (III) exploration of the relationship between the methodological and reporting quality and assessment of potential factors influencing the quality of MAs related to SCT. Based on the aforementioned aims, the research status quo and evidence gap of MAs concerning SCT will be presented completely. We present the following article in accordance with the Preferred Reporting Items for Systematic Review and Meta-analysis Protocols (PRISMA-P) reporting checklist (available at http://dx.doi.org/10.21037/atm-20-6892) (20).

Methods

The overview of published MAs concerning SCT will be conducted by referencing the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) guideline (21) and Cochrane handbook (22). Meanwhile, we have registered this protocol on PROSPERO website (registration number: CRD42020206642). In addition, for any deviations from this protocol, we will report the details and reasons for any change in the following full-text.

Search methods

Two largest and most frequently used English databases, including PubMed and EMBASE, will be systematically searched from inception to October 2020. We used the subject headings and keywords to establish our search strategy, with the main search words including: “stem cells”, “mesenchymal stem cells”, “stem cell transplantation”, “peripheral blood stem cell transplantation”, “cord blood stem cell transplantation”, “hematopoietic stem cell transplantation”, “systematic review”, “meta-analysis” and “meta”. There will be no other language or publication type restrictions. In addition, we will check reference lists of included MAs for retrieving other potentially appropriate MAs. The search details are presented in Supplementary file.

Eligibility criteria and study selection

We will include the MAs that fit the following criteria: (I) study type: traditional MAs of clinical and preclinical primary studies investigating the efficacy and (or) safety of SCT vs. any other interventions, published in English; the definition of MA is according to the Cochrane handbook “The statistical combination of at least two studies to produce a single estimate of the effect of the healthcare intervention under consideration. A MA is simply the statistical combination of research from studies” (22); (II) subject: human or animal models with any condition, e.g., chronic liver disease (12), stroke (13), and sepsis (14); (III) intervention: all available SCTs with different delivery routes, including but not limited to, mesenchymal stem cell and adipose-derived stem cell; (IV) control: any comparison group, e.g., conventional treatment or placebo; (V) outcome: any outcomes that involve efficacy and untoward reactions, e.g., pain reduction (5) and mortality (14). Studies will be excluded if we could not access the key data or the full-text of MAs; the study type is a protocol, comment, letter, or conference abstract; or if the study is a network MA, individual patient data MA, qualitative systematic review, scoping literature review, or traditional expert review. All identified records will be imported into the reference manager software EndNote X9 (Thomson Corporation, Thomson ResearchSoft, USA) to select eligible MAs according to the abovementioned eligibility criteria. While after deduplication, two independent reviewers will perform the literature selection by reading the titles and abstracts, and the full-text of potentially appropriate MAs will be downloaded for further assessing their eligibility. In case of any discrepancy in this process, two independent reviewers will reach a consensus through discussion, or by consulting a third reviewer to make the final decision.

Quality assessment

For completely mapping the quality of MAs focusing on SCT, the two most frequently used tools, PRISMA and A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR-2) (23) will be used to appraise the reporting quality and methodological quality, respectively. PRISMA was designed to improve the reporting quality of MAs, and this guideline consists of 27 items. In this overview, the reporting quality of MAs will be assessed based on the contents in the document reported by the authors of MAs, and each item will be answered as “Yes” (the study fully met the item requirements), “Partial Yes” (the study incompletely met the item requirements), or “No” (the study did not meet the item requirements). AMSTAR-2 contains 16 items and is an updated tool for original AMSTAR. Seven of the 16 items in this tool (items 2, 4, 7, 9, 11, 13, and 15) are critical items based on the report of this tool. Similarly, three options including “Yes”, “Partial Yes”, or “No” will be used to complete the methodological quality assessment, and the overall confidence or methodological quality of a specific MA will be classified by assessing the critical and noncritical items into four levels: high (a study with no more than one noncritical weakness), moderate (a study with more than one noncritical flaws), low (a study with one critical weakness, and with or without a noncritical weakness), very low (a study with more than one critical flaws, and with or without a noncritical weakness). Two experienced reviewers will independently complete the process of quality assessment; any conflict will be solved through a discussion. A third reviewer will be consulted when necessary.

Data extraction

Two reviewers will independently extract data from each MA using a predesigned and piloted data extraction form. Any discrepancy will be solved through a consensus following a discussion. The key data that will be extracted will include the following: title, first author, number of authors, clinical topics or category of diseases, research aim, subject type (human, animal models, or a combination), year of publication, registration information, origin or country (the country of the corresponding author will be considered as the origin/country of a specific MA), journal name and impact factor, number of pages, the description of intervention details (i.e., information on SCT and comparison with control), database or source searched in the MA, search timespan, sample size of the MA (i.e., number of subjects included in each MA), number of primary studies, the tool for the assessment of risk of bias or methodological quality, funding information, involvement of a methodologist (e.g., epidemiologists and statistician, deciding by the institutes conducting the MA), publication bias assessment, use of GRADE (Grading of Recommendations, Assessment, Development and Evaluation) (6), subgroup analysis, sensitivity analysis, and meta-regression, efficacy and safety outcomes with effect sizes reported in the MAs publications, key findings and conclusion.

Data analysis

A detailed qualitative description and quantitative analysis will be used in this overview. The efficacy and safety data concerning SCT reported in MAs will be summarized descriptively. The compliance rate for items of PRISMA and AMSTAR-2 will be calculated, including the numbers and their percentage of “Yes”, “Partial Yes”, or “No”. The relationship between reporting quality and methodological quality will be assessed using Spearman’s rank correlation coefficient based on the number of “Yes” response (24). Similarly, according to the number of full compliance (“Yes” response), seven factors that potentially influence the reporting quality and methodological quality [i.e., publication year, impact factor, number of pages, registration (yes/no), international cooperation (yes/no), funding (yes/no), and involvement of a methodologist (yes/no)] will be explored using univariate and multivariate linear regression analyses. The variance inflation factor (VIF) will be used to assess the multicollinearity, and the threshold will be set as 6 in our study the accepted threshold value in other relevant publications (i.e., VIF <6 is acceptable) (25). Moreover, after summing the number of “Yes” and “Partial Yes” responses, we will conduct sensitivity analyses to consider the impact of “Partial Yes” response for the results on the relationship between reporting quality and methodological quality and influencing factors exploration. In addition, the evidence mapping method (26,27) will be used to present the results for quality assessment. The evidence mapping will constitute two axes and some bubbles. The X axe will be divided into four parts corresponding to the results concerning the methodological quality (“High”, “Moderate”, “Low”, and “Very low”) of AMSTAR-2 for each MA, whereas the Y axe will be used to describe the percentage of full compliance (“Yes” response) with the items of PRISMA. Each bubble will be used to represent a specific MA. The size of the bubble will be proportioned to the sample size of the MA, the color of the bubble will be used to describe the type of MAs (red for clinical only, green for preclinical only, and yellow for a combination). All data analyses will be completed using Stata 16.0 (StataCorp, College Station, TX, USA) and Excel 2016 (Microsoft Corporation, WA, USA). P<0.05 will be considered statistically significant.

Discussion

SCT is an emerging treatment method for many diseases, e.g., chronic liver disease, diabetes mellitus, and knee osteoarthritis. Therefore, it offers huge hope for patients with these conditions. However, the clinical decision-making should be based on high-quality evidence and should weigh benefits and harms. Although many MAs (3,5,12-14) related to SCT for various diseases have been published, their quality and clinical outcomes are unknown. Therefore, we designed an overview to map the reporting quality and methodological quality and clinical effects reported in MAs concerning SCT. To the best of our knowledge, this overview of MAs constitutes the first attempt to appraise the quality of MAs focusing on SCT based on PRISMA and AMSTAR-2 tools. Moreover, the overview will be the first to provide the clinical and preclinical evidence of efficacy and safety of SCT in MAs. Nonetheless, our study has some limitations. The most noteworthy limitation of our study is that we will only search the two largest databases and only English publications will be included. Therefore, it is possible for us to miss some studies, but hand-checking reference lists of the included MAs may compensate for this shortcoming and decrease this bias. In our overview, we will also explore the association between reporting quality and methodological quality, and assess the potential factors influencing the overall quality. Hopefully, the results of this overview of MAs concerning SCT will be useful for informed decision-making in a clinical setting and aiding further scientific research.

Presenting and reporting of results

The PRISMA guideline (21) will be used to support the reporting of the final full-text. The results of this overview of MAs concerning SCT will be submitted to a peer-reviewed journal for publication.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the PRISMA-P reporting checklist. Available at http://dx.doi.org/10.21037/atm-20-6892

Peer Review File: Available at http://dx.doi.org/10.21037/atm-20-6892

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-6892). The authors have no conflicts of interest to declare.

Ethical Statement: All authors are accountable for all aspects of this work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study will be completed based on published literature; therefore, research ethics board approval and consent are waived.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Blau HM, Daley GQ. Stem Cells in the Treatment of Disease. N Engl J Med 2019;380:1748-60. [Crossref] [PubMed]
2. Sneddon JB, Tang Q, Stock P, et al. Stem Cell Therapies for Treating Diabetes: Progress and Remaining Challenges. Cell Stem Cell 2018;22:810-23. [Crossref] [PubMed]
3. Zhang Y, Chen W, Feng B, et al. The Clinical Efficacy and Safety of Stem Cell Therapy for Diabetes Mellitus: A Systematic Review and Meta-Analysis. Aging Dis 2020;11:141-53. [Crossref] [PubMed]
4. Kenihan L, McTier L, Phillips NM. Patients' expectations and experiences of stem cell therapy for the treatment of knee osteoarthritis. Health Expect 2020;23:1300-9. [Crossref] [PubMed]
5. Kim SH, Djaja YP, Park YB, et al. Intra-articular Injection of Culture-Expanded Mesenchymal Stem Cells Without Adjuvant Surgery in Knee Osteoarthritis: A Systematic Review and Meta-analysis. Am J Sports Med 2020;48:2839-49. [Crossref] [PubMed]
6. Zähringer J, Schwingshackl L, Movsisyan A, et al. Use of the GRADE approach in health policymaking and evaluation: a scoping review of nutrition and physical activity policies. Implement Sci 2020;15:37. [Crossref] [PubMed]
7. Lu C, Lu T, Ge L, et al. Use of AMSTAR-2 in the methodological assessment of systematic reviews: protocol for a methodological study. Ann Transl Med 2020;8:652. [Crossref] [PubMed]
8. Lund H, Brunnhuber K, Juhl C, et al. Towards evidence based research. BMJ 2016;355:i5440. [Crossref] [PubMed]
9. Pussegoda K, Turner L, Garritty C, et al. Systematic review adherence to methodological or reporting quality. Syst Rev 2017;6:131. [Crossref] [PubMed]
10. Luo YN, Zheng QH, Liu ZB, et al. Methodological and reporting quality evaluation of systematic reviews on acupuncture in women with polycystic ovarian syndrome: A systematic review. Complement Ther Clin Pract 2018;33:197-203. [Crossref] [PubMed]
11. Zhu X, Shen X, Hou X, et al. Total glucosides of paeony for the treatment of rheumatoid arthritis: A methodological and reporting quality evaluation of systematic reviews and meta-analyses. Int Immunopharmacol 2020;88:106920. [Crossref] [PubMed]
12. Zhou GP, Jiang YZ, Sun LY, et al. Therapeutic effect and safety of stem cell therapy for chronic liver disease: a systematic review and meta-analysis of randomized controlled trials. Stem Cell Res Ther 2020;11:419. [Crossref] [PubMed]
13. Lalu MM, Montroy J, Dowlatshahi D, et al. From the Lab to Patients: a Systematic Review and Meta-Analysis of Mesenchymal Stem Cell Therapy for Stroke. Transl Stroke Res 2020;11:345-64. [Crossref] [PubMed]
14. Sun XY, Ding XF, Liang HY, et al. Efficacy of mesenchymal stem cell therapy for sepsis: a meta-analysis of preclinical studies. Stem Cell Res Ther 2020;11:214. [Crossref] [PubMed]
15. Johnson WD, Rivadeneira N, Adegbite AH, et al. Human Immunodeficiency Virus Prevention for People Who Use Drugs: Overview of Reviews and the ICOS of PICOS. J Infect Dis 2020;222:S278-300. [Crossref] [PubMed]
16. Bonetti AF, Della Rocca AM, Lucchetta RC, et al. Mapping the characteristics of meta-analyses of pharmacy services: a systematic review. Int J Clin Pharm 2020;42:1252-60. [Crossref] [PubMed]
17. Hoffmann TC, Oxman AD, Ioannidis JP, et al. Enhancing the usability of systematic reviews by improving the consideration and description of interventions. BMJ 2017;358:j2998. [Crossref] [PubMed]
18. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009;374:86-9. [Crossref] [PubMed]
19. Grainger MJ, Bolam FC, Stewart GB, et al. Evidence synthesis for tackling research waste. Nat Ecol Evol 2020;4:495-7. [Crossref] [PubMed]
20. Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ 2015;350:g7647. [Crossref] [PubMed]
21. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009;339:b2535. [Crossref] [PubMed]
22. Higgins JPT, Thomas J, Chandler J, et al (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). Cochrane 2020. Available online: www.training.cochrane.org/handbook
23. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017;358:j4008. [Crossref] [PubMed]
24. Chapman SJ, Drake TM, Bolton WS, et al. Longitudinal analysis of reporting and quality of systematic reviews in high-impact surgical journals. Br J Surg 2017;104:198-204. [Crossref] [PubMed]
25. Li L, Xu C, Deng K, et al. The reporting of safety among drug systematic reviews was poor before the implementation of the PRISMA harms checklist. J Clin Epidemiol 2019;105:125-35. [Crossref] [PubMed]
26. Madera Anaya M, Franco JVA, Ballesteros M, et al. Evidence mapping and quality assessment of systematic reviews on therapeutic interventions for oral cancer. Cancer Manag Res 2018;11:117-30. [Crossref] [PubMed]
27. Lunny C, Brennan SE, McDonald S, et al. Evidence map of studies evaluating methods for conducting, interpreting and reporting overviews of systematic reviews of interventions: rationale and design. Syst Rev 2016;5:4. [Crossref] [PubMed]