A scientificity and feasibility evaluation of COVID-19 clinical studies registered in China

Introduction

On December 29, 2019, a severe pneumonia cluster case with unknown cause was reported in a hospital in Wuhan, and the Chinese government notified the World Health Organization (WHO), after verification (1). On January 7, 2020, Chinese researchers isolated the virus from the patient’s lower respiratory tract, the pathogen of the outbreak was confirmed as a novel coronavirus (2019-nCoV) and named coronavirus disease 2019 (COVID-19) by the WHO on February 12, 2020 (2). The epidemic started in China and spread into a large-scale outbreak across other parts of the world. On March 11, 2020, the WHO declared the outbreak to be a global pandemic (3). To date, 638,146 COVID-19 cases and 30,105 deaths have been confirmed globally (4).

As an emerging disease, COVID-19 is still in the early stages being understood, and there are a series of questions to be answered in terms of diagnosis, treatment, and prognosis assessment. The Chinese government has published seven versions of the treatment plan to fight the epidemic, which has become an essential part of the Chinese initiative against COVID-19 (5). However, the methods to improve the treatment plan, further improve efficacy, and reduce the case fatality rate require more scientific evidence. Scientists from all over the world have conducted or are conducting in-depth scientific research on epidemic prevention and control. On March 13, 2020, the WHO announced a total of 1,584 academic papers had been published globally on COVID-19, including narrative reviews or opinions (814 articles), case studies/series (213 articles), and epidemiological studies (124 articles) (6). As the virus first outbreak in China, Chinese scientists have been able to carry out many clinical studies due to having access sooner to a higher number of cases. Some registered clinical studies have reported on drugs and other treatments concerning COVID-19 (7,8). However, whether the research protocols of these studies are scientific and feasible has not yet been examined. Therefore, our research group used the registration data of the clinical research registration platform to collate and analyze the registered research protocols, evaluate these research protocols concerning interventional research, and make suggestions for the design and smooth implementation of clinical research.

Methods

Data sources and retrieval

On March 11, 2020, we performed a search on the International Clinical Trials Registry Platform (ICTRP) of the WHO and clinicaltrials.gov database, with keywords including “Covid-19”, “COVID,” “2019-novel Corona Virus (2019-ncov)”, “Novel Coronavirus Pneumonia (NCP)”, “Severe Acute Respiratory Infection (SARI)”, “Severe Acute Respiratory Syndrome-Corona Virus-2 (sars-cov-2)”, “nCoV”, “Coronavirus”, and “Severe Acute Respiratory Syndrome (SARS)”.

Inclusion and exclusion criteria

To filter the articles retrieved, we set the inclusion criteria to be all registered studies of the novel Corona Virus for 2019, while the exclusion criteria included duplicated studies, canceled studies, non-population-based studies, and studies on other viruses.

Research screening and data extraction

Research screening and data extraction were performed by four researchers in the public health profession. All the researchers involved in the research, screening and data extraction were trained and began extracting the data after completing the training. The extracted data included general information (research type, registration time, primary sponsor, centers of research implementation, fund support, ethical status) and study design (grouping, sample size, randomization, blind, main efficacy indicators, etc.). If there were questions concerning the data extraction, these would be discussed with senior researchers.

Statistical analysis

SPSS 24.0 (IBM Corp., NY, USA) was used for all statistical analyses. Categorical variables were summarized as the counts (percentages). Continuous variables were expressed as medians (25th percentile, 75th percentile) with the abnormal distribution. Since there were no statistical inferences in this paper, no statistical tests were performed.

Results

Basic registration information

As of March 11, 2020, 330 records were obtained from the ICTRP, and 145 records were obtained from the ClinicalTrials.gov website. After merging the two datasets, 19 duplicate records were excluded according to their registration ID, 56 records of non-novel coronavirus researches were excluded according to the title and research purpose, 4 records were excluded according to non-population-based studies, and 17 (4.3%) records were excluded as canceled studies (Figure 1). A total of 379 records were included in this study (Table 1). Therapeutic research accounted for 68.6% of patients, and 367 (96.8%) registered studies being conducted by research leaders in China. As of March 11, 2020, only 2 (0.5%) studies have been completed.

Figure 1 The flow chart of registered studies retrieval and screening.

Table 1 Basic information for all registered studies
Full table

Distribution of registration number over time

The first clinical study was registered online on January 23 with a randomized controlled trial (RCT) design. This was only 2 weeks after the coronavirus was identified, and 554 cases were confirmed in China that day. Chinese research focused on clinical research fairly earlier and moved rapidly, considering the need or registration review, research program formulation, and ethical issues. The days with the highest number of studies registered were February 18 and 19, 2020 (both with 22 studies registered). The specific registration date distribution is shown in Figure 2. The trend of interventional and observational studies is the same, but the number of interventional studies is greater than observational studies (263 vs. 116).

Figure 2 The number of registered COVID-19 studies in the International Clinical Trials Registry.

Basic information on interventional research in China

Since interventional studies need to be registered while observational studies do not, the following analysis only selected the interventional studies conducted in China, with a total of 257 studies included (Table 2).

Table 2 Basic information for interventional study registered in China
Full table

Therapeutic research accounted for 91.1% of cases. Among the available registered research, 79.6% of the studies had been given ethical approval, and national or local longitudinal funds supported. Pharmaceutical companies have also provided greater support in the research. Nearly 60% of the studies included all types of patients, only 16% included common cases, and nearly 20% included severe cases only. The subjects were divided into two groups in 70% of the studies, while there were 15 studies with more than 4 groups.

Scientificity of the study protocol

In nearly 90% of studies with two groups or more, the subjects were randomly assigned. Nearly 15% of all interventional studies used blinding in the study design. For the outcome indicators, 1–2 primary outcomes indicators are generally recommended, but 35% of the studies have more than 2 indicators. The study with the highest number of indicators included 37 primary outcomes indicators, and 11 studies had more than 10 primary outcomes indicators. The primary indicators were objective indicators in 80% of the studies.

Feasibility of the study protocol

The institution of primary sponsor does not admit cases in nearly 25% of the studies, so there is a question to be answered whether a principal investigator can manage and control the unit of data collection well. Concerning the study sample size, half of the research was conducted with sample sizes of between 60 and 200 individuals. We could not determine the calculation method of the sample size, but sample sizes in RCT studies deserve careful consideration as the results may not meet statistical requirements. Based on the existing literature data, it is assumed that a total of at least 422 subjects would be needed for the outcome of mortality, and a total of at least 128 subjects would be needed for the outcome of the severe cases proportions (Table 3). Only 14 (5.4%) and 92 (35.8%) studies met the sample size requirement on mortality and severe case, respectively, and thus most studies did not meet sample size requirements. We calculated cases needed for the studies, and the total was 32,333 cases, which accounts for about 40% of all confirmed cases in China.

Table 3 Simulated sample size calculation based on different outcomes
Full table

Discussion

When an unknown disease occurs, how our researchers respond is an important question. In response to the COVID-19 epidemic, scientists worldwide are working diligently to explore and help prevent and control the epidemic scientifically. At present, more than 2,800 academic papers have been published worldwide (6). We have conducted a comprehensive review and summary of the research registration scheme for this disease, laying the foundation for a scientific response. Our researchers responded quickly to this disease, starting the research at an early stage of the disease, and registered studies have also been vast, which shows the positive scientific attitude of the researchers in dealing with the disease. Simultaneously, administrative and commercial companies have also deployed various funds to encourage researchers to carry out scientific exploration, with half of the research conducted having been supported by these funds.

What research should be prioritized when a new disease occurs? Some researchers have pointed out that most important research questions should concern virus transmission, asymptomatic and presymptomatic virus shedding, diagnosis, treatment, vaccine development, the origin of the virus, and viral pathogenesis (9). Among the types of research registered, therapeutic research, preventive investigation, and diagnostic research are the three most common types. Therapeutic research accounts for nearly 70% of these studies, indicating that our greatest need is the treatment of infectious diseases. In the case of a surge in the number of infected people, it is imperative to find the most appropriate treatment for prevention and control. Other preventive investigation studies can demonstrate the infectious characteristics of diseases utilizing infectious disease investigation and provide a basis for formulating prevention and control strategies to cut off transmission routes. As nucleic acid detection and diagnosis methods had been developed before this outbreak (10), there are not many diagnostic studies, but even with the rapid diagnosis and accurate classification of patients, many problems still need to be solved.

To date, 89% of the research has been designed with a parallel comparison. Among these parallel comparison study designs, 88% adopted randomization, 14% adopted blinding, and 80% used objective indicators. Thus, it can be stated the clinical studies have been conducted scientifically, and have used methods and technical means to ensure reproducible results and minimization of bias. Meanwhile, some RCTs have obtained important results (11,12). Cao and colleagues’ trial showed no benefit was observed with lopinavir-ritonavir treatment beyond standard care in severe COVID-19 patients (11).

In terms of feasibility, the protocol of these studies shows that there may be major deficiencies. Most researchers from Beijing and Shanghai led some studies, but they did not have enough patients to study in their cities. If the main research units do not participate in the collection of cases, brings significant challenges to the management and control of the research implementation process (13). Second, the sample size estimation method of these studies has not been made available, but, for the case fatality rate of the disease and the proportion of severe to mild cases, at least 70% of the studies may not meet sample size requirements. The total sample size has reached 40% of the cases confirmed in China. Most of the studies have not been completed, or the collection of data has not been started, which may result in subsequent studies being incomplete. Therefore, we suggest that in the face of major emerging infectious diseases, our research management department should actively manage studies, select scientific and feasible programs, and focus on quicker research, to avoid duplication and wasting research efforts on unnecessary interventions. Also, we can select research that is of adequate quality and which has a satisfactory preliminary result, to carry out international multicenter studies, and continue to optimize the efficiency of research to avoid the waste of experimental resources.

It is also important to note all new clinical trials in which enrollment began on or after January 1, 2019, which must include a data sharing statement in the trial’s publicly posted registration (14). In response to the COVID-19 epidemic, rapid data sharing is the basis for more effective public health action. Efforts for expedited data and results reporting should not be limited to clinical trials but should include observational studies and operational research (15). Indeed, the timely dissemination of scientific information is an effective way to reduce public panic about COVID-19 (16).

Some limitations of this study should also be addressed. Firstly, although clinical research registration has been implemented for many years, some clinical investigators may not register all studies on the Internet, and the collection of our data might thus have been biased to some extent. Secondly, certain registration information, especially related to the sample size calculations and randomization methods, is incomplete or not required in the registry; therefore, our research quality and feasibility evaluation might have been subject to certain restrictions. This also suggests we should strengthen the supervision and quality control of future clinical trial registration, avoid considering low-quality studies with insufficient information for registration, and improve the quality of clinical research registration platforms.

Conclusions

Registrations for clinical studies on COVID-19 has been substantial, the response of researchers has been fast, and the study design quality has been high. However, there are also shortcomings in general, such as insufficient sample size, poor coordination among multidisciplinary teams, and weak quality control in the research process.

Acknowledgments

Funding: This work was supported by the National Key R&D Program of China (2018YFC1704400).

Footnote

Data Sharing Statement: Available at http://dx.doi.org/10.21037/atm-20-2943

Peer Review File: Available at http://dx.doi.org/10.21037/atm-20-2943

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-2943). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. China CDC, Epidemic Update and Risk Assessment of 2019 Novel Coronavirus-China. January 28, 2020.
2. Zhu N, Zhang D, Wang W, et al. A Novel Coronavirus from Patients with Pneumonia in China, 2019. N Engl J Med 2020;382:727-33. [Crossref] [PubMed]
3. World Health Organization. Coronavirus disease (COVID-19) Pandemic. Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019
4. World Health Organization. Coronavirus disease 2019 (COVID-19) Situation Report-60. Available online: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200320-sitrep-60-covid-19.pdf?sfvrsn=d2bb4f1f_2
5. National Health Commission of The People’s Republic Of China. New coronavirus pneumonia diagnosis and treatment plan (trial version 7). Available online: http://www.nhc.gov.cn/yzygj/s7653p/202003/46c9294a7dfe4cef80dc7f5912eb1989/files/ce3e6945832a438eaae415350a8ce964.pdf
6. World Health Organization. Global research on coronavirus disease (COVID-19). Available online: https://www.who.int/emergencies/diseases/novel-coronavirus-2019/global-research-on-novel-coronavirus-2019-ncov
7. Zhang Q, Wang Y, Qi C, et al. Clinical trial analysis of 2019-nCoV therapy registered in China. J Med Virol 2020;92:540-5. [Crossref] [PubMed]
8. Rosa SGV, Santos WC. Clinical trials on drug repositioning for COVID-19 treatment. Rev Panam Salud Publica 2020;44:e40. [Crossref] [PubMed]
9. Yuen KS, Ye ZW, Fung SY, et al. SARS-CoV-2 and COVID-19: The most important research questions. Cell Biosci 2020;10:40. [Crossref] [PubMed]
10. Adhikari SP, Meng S, Wu YJ, et al. Epidemiology, causes, clinical manifestation and diagnosis, prevention and control of coronavirus disease (COVID-19) during the early outbreak period: a scoping review. Infect Dis Poverty 2020;9:29. [Crossref] [PubMed]
11. Cao B, Wang Y, Wen D, et al. A Trial of Lopinavir-Ritonavir in Adults Hospitalized with Severe Covid-19. N Engl J Med 2020;382:1787-99. [Crossref] [PubMed]
12. Gautret P, Lagier JC, Parola P, et al. Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial. Int J Antimicrob Agents 2020. [Crossref] [PubMed]
13. Gentry KR, Arnup SJ, Disma N, et al. Enrollment challenges in multicenter, international studies: The example of the GAS trial. Paediatr Anaesth 2019;29:51-8. [Crossref] [PubMed]
14. Statham EE, White SA, Sonwane B, et al. Primed to comply: Individual participant data sharing statements on ClinicalTrials.gov. PLoS One 2020;15:e0226143. [Crossref] [PubMed]
15. Moorthy V, Henao Restrepo AM, Preziosi MP, et al. Data sharing for novel coronavirus (COVID-19). Bull World Health Organ 2020;98:150. [Crossref] [PubMed]
16. Song P, Karako T. COVID-19: Real-time dissemination of scientific information to fight a public health emergency of international concern. Biosci Trends 2020;14:1-2. [Crossref] [PubMed]