Appealing for efficient, well organized clinical trials on COVID-19

Introduction

As of February 29, 2020, approximately 85,000 cases of coronavirus disease 2019 (COVID-19) have been confirmed worldwide, with nearly 3,000 deaths occurred. Recently, many COVID-19-related interventional clinical trials have emerged in China, a high-incidence area of COVID-19 (1). The study aims of these interventional trials include eradicating virus, relieving pneumonia symptoms and promoting recovery, etc. The rapid emergence of these trials stimulated a wave of discussion on whether it is appropriate to start so many clinical trials at the critical time point of the battle with the coronavirus.

Recently, a group of Chinese scientists made an urgent call for raising the scientific rigorousness of clinical trials on COVID-19 (2). In order to understand the characteristics of the ongoing or pending interventional clinical trials on COVID-19, we reviewed the information of these trials registered at Chinese Clinical Trial Registration (ChiCTR) or ClinicalTrials.gov (3,4).

Methods

The data of interventional trials from ChiCTR and ClinicalTrials.gov were retrieved updated on February 22, 2020. Two authors (Zhao and Shen) independently extracted the relevant information from the downloaded records, including study phase, study design, presence or absence of randomization, blinding, sample size, severity of disease and source of samples.

To assess the quality of these trials, we used a self-developed scale which involved the applicable items from criteria selected from the Jadad score, Delphi and Cochrane Collaboration (5). The scale includes 7 questions:

• Was the study described as randomized?
• Was the method used to generate the sequence of randomization described and appropriate?
• Were the eligibility criteria specified?
• Were the patients blinded to the treatment?
• Were the care provider and investigator blinded to the treatment?
• Did the study provide primary efficacy endpoint?
• Did the study provide safety endpoint?

We counted the number of positive responses to each question as the score of the quality of a trial.

Descriptive statistics (frequencies and proportions) are used to present the characteristics of the trials. All analyses were performed using R (version 3.6.3).

Results

A total of 171 COVID-19-related interventional trials were identified (138 from ChiCTR and 33 from ClinicalTrials.gov). The registration date distribution of these trials was shown in Figure 1. Of the trials registered at ChiCTR, 120 were approved by the institutional review boards. The first 2 trials were registered on Jan 23, 2020, the first day of the “lockdown” of Wuhan city.

Figure 1 The registration date distribution of the interventional clinical trials.

The 171 trials are classified into 4 categories based on treatment modalities, including chemical drugs (CDs), biological therapies (BTs), traditional Chinese medicine (TCM) treatments and other therapies. The biological treatments include PD1, Bevacizumab or other monoclonal agents, cell treatments (e.g., NK cells) and convalescent plasma therapy. TCM treatments include both Chinese medicines (TCMs) and Chinese physical treatments, such as “Qigong” and “Taiji”. The characteristics of these trials were summarized in Table 1. The mean quality scores for CDs, BTs, TCMs and other therapies are 4.5, 4.0, 4.2 and 4.1, respectively.

Table 1 Characteristics of the 171 interventional clinical trials [n (%)]
Full table

There are 11 confirmatory Phase III or Phase II/III studies (6.4%), 8 of which are for CDs including Remdesivir, Ritonavir, Triavevirin, Leflumite, Glucocorticoid, Hydroxychloroquine, Darunavir and Cobicistat. Two Phase III trials for biological treatments investigate the effect or Bevacizumab or Intravenous Immunoglobulin. There is also a Phase III trial on Yinhu Qingwen Decoction, a combination of 12 TCMs.

Most trials (88.3%) use parallel-arm design. Randomization is applied in 132 trials (77.2%), while blinding in only 25 trials (14.6%). The proportion of using blinding (single, double, triple or quadruple) is higher for trials on CDs (24.6%) than on the others.

The sample sizes of 45.6% trials are more than 100. Two trials on TCM treatments are planned to recruit 800 or more patients. Severe and critically ill patients are respectively recruited in 70 (40.9%) and 28 (16.4%) trials. More than half of the trials (72.4%) are planned to collect blood samples, while only 19 trials (13.8%) to collect nasopharyngeal swabs.

Most primary endpoints (PEs) are efficacy-related, while only 1 trial also included safety in the PEs. There are 113 trials (66.1%) including safety as secondary endpoint. More trials on TCM treatment use 2 or more PEs than those on CDs or BTs (57.6%, 39.4% and 40.5%, respectively). Some trials even have 6 or more PEs. The most 3 commonly used PEs are associated with clinical symptoms (73.1%), pathogen (29.2%) and lung function (27.5%). Trials on CDs have greater proportions of pathogen (41.0%) or fatality (18.0%) related PEs than the others. It is worth noting that these trials have registered over 300 distinct PEs totally.

Discussion

During the COVID-19 outbreak, we are in urgent need of effective treatment strategies. Through analyzing currently registered interventional trials, we found some studies with potential defects including unreasonable design, inappropriate PE definition, small sample size and ethical issue.

There are 17 trials (9.9%) using single arm designs. Without an appropriate control arm, we cannot exclude the possibility that the recovery of patients is due to the natural progress of the disease or other concomitant therapies. There are 132 trials registered as randomized trials. Randomization is a widely used technique in clinical trials to remove both measured and un-measured confounding effects. Without randomization, the “causal” treatment effect cannot be estimated as we are not sure whether there are un-measured confounders possibly biased the analysis. Only 17 trials used double, triple or quadruple blinding, while 8 trials used single blinding. As human behavior is influenced by what they know or believe, it is important to use blinding (sometimes called masking) to remove the bias resulted from the subjectivity in assessment. Recently, Cai et al. reported the result from an open-label control study on the efficacy of Favipiravir (FPV) for COVID-19 (6). Although their findings demonstrated a positive improvement on the viral clearance of FPV than the control therapy, the authors acknowledged the limitations that the study “is not a randomized, double-blind, parallel trial, further well-designed and large-scale confirmatory trials are warranted”. Methods of sensitivity analysis, such as confounding functions or bounding factors, can be applied when analyzing data from trials with potential bias (7).

Determining the appropriate PEs is crucial to address the primary scientific question of a clinical trial. About one third trials used more than 2 PEs. Trials on TCMs were more likely to use more than 2 PEs than trials on CDs or BTs. Using too many PEs in a trial may inflate the type I error rate. As an example, we noticed that some trials registered 10 PEs. Suppose that the significance level for each PE is 0.05, the overall type I error rate of that trial would be 1-(0.95)¹⁰ 0.4, which is far more beyond the level that we can tolerate, especially in confirmatory trials. Some may argue that using more PEs may increase the probability to identify a useful treatment. However, the possibility of false discovery is also increased. Forwarding too many useless treatments to late phase trials, would exhaust the already limited medical resources. Also, the PE should be defined clearly and specifically to improve the operation quality. Objective endpoints, such as pathogen or imaging results, are preferred to avoid the bias during evaluation. We also noticed that it is possible that some investigators confused the definitions of PE and measurement of efficacy.

It is possible that some trials lack statistical power. Assuming a two-side significance level of 0.05 and a power of 80%, a trial on mild pneumonia patients should recruit nearly 1,000 patients if the effective rate increases from 90% to 95% for the new treatment. More patients are needed for the trials planning to decrease the fatality of critically ill patients. Many trials have high probabilities of failing as most registered trials only try to recruit less than 400 patients. Given that there are less and less confirmed COVID-19 patients in China, these powerless trials would exhaust limited patients resources.

Some drugs or therapies under study are short of previous theoretical support for the treatment of COVID-19. As an example, a trial which would recruit more than 200 patients tried to investigate the effect of “Qigong” on the recovery of lung functions for critically ill patients. There is no solid evidence that these therapies may be beneficial to the patients, leading to non-compliance to ethical standards. These unnecessary trials may also waste medical resources, as well as diverting patients’ resources. Recently, two trials on the efficacy of Remdesivir on critically ill Chinese patients had been terminated due to the lack of patients. WHO has suggested an establishment of a centralized research program to ensure the most promising researches (8).

During the preparation of this article, we have noticed that while the number of registered trials keep rising (ChiCTR: 282 on April 18, 2020), only 1,734 confirmed COVID-19 patients are still under treatment as on April 18, 2020. On April 6, 2020, the state council of the People’s Republic of China has issued a file urging the medical institutions to regulate the clinical trials for COVID-19. Clinical trials on COVID-19 should be designed based on scientific rules (appropriate controls, randomization, blinding, and sufficient sample size), ethics and patients’ benefits. The standards for clinical trials should not be compromised even in such a special time.

Acknowledgments

The authors are grateful to Miss Yuanping Yue and Mrs. Yue Wang for their help on the information extraction during the preparation of the manuscript.

Funding: This study was funded by the Special Program of National Natural Science Foundation of China on tracing, pathogenesis, prevention and treatment for COVID-19 (No. 82041024 to FC, the National Natural Science Foundation of China (No. 81872709 to YZ) and Key Project of the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (No. 18KJA110004 to YZ).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-2429). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Maxmen A. More than 80 clinical trials launch to test coronavirus treatments. Nature 2020;578:347-8. [Crossref] [PubMed]
2. Chen F, Hao YT, Zhang ZJ, et al. An urgent call for raising the scientific rigorousness of clinical trials on COVID-19. Zhonghua Liu Xing Bing Xue Za Zhi 2020;41:301-2. [PubMed]
3. Chinese Clinical Trial Registration. Available online: http://www.chictr.org.cn. Accessed February 22 2020.
4. ClinicalTrials.gov. Available online: https://clinicaltrials.gov/. Accessed February 22 2020.
5. Berger VW, Alperson SY. A general framework for the evaluation of clinical trial quality. Rev Recent Clin Trials 2009;4:79-88. [Crossref] [PubMed]
6. Cai Q, Yang M, Liu D, et al. Experimental Treatment with Favipiravir for COVID-19: An Open-Label Control Study. Engineering 2020. [Crossref] [PubMed]
7. Kasza J, Wolfe R, Schuster T. Assessing the impact of unmeasured confounding for binary outcomes using confounding functions. Int J Epidemiol 2017;46:1303-11. [Crossref] [PubMed]
8. WHO. Report of the WHO-China Joint Mission on Coronavirus Disease 2019 (COVID-19) 2020 February 24.