Impact of the COVID-19 pandemic on clinical trials: a cross-sectional questionnaire study in China

Introduction

In early 2020, the coronavirus disease 2019 (COVID-19) started to wreak havoc in various regions across the world. Despite the measures taken by all countries to combat the pandemic, the novel coronavirus has been characterized by its rapid mutation and transmission, which means that the progression of the pandemic is highly uncertain. In many countries, healthcare infrastructural systems have experienced challenges, with even basic and routine patient care not being guaranteed (1), and clinical research has inevitably been undermined, with the integrity and rigor of clinical trials being particularly affected (2,3). During March 2020, fewer than 20% of institutions in the United States and Europe reported enrolling patients at a usual rate (4,5). Registration data in the National Library of Medicine (ClinicalTrials.gov) indicate that due to COVID-19, 1,130 trials were suspended from April 2020 to March 2021, and the rollout of COVID-19 vaccinations did not change the increasing number of suspended trials (6). Furthermore, the outbreak of the pandemic has also inevitably affected clinical trials in China, a country which contributes about one-fifth of global clinical trials, and from which the trials are increasing year by year (7,8).

Although there are continuously sporadic cases of COVID-19 in some cities, Chinese residents have generally resumed work and life since April 2020, and the medical infrastructure has returned to pre-pandemic levels in most regions (9,10); however, the conduct of clinical research has changed somewhat. Many regulatory and research organizations, for example, the US Food and Drug Administration (FDA) and European Medical Agency (EMA), issued special guidance and developed new policies and procedures to address the conduct of clinical trials during the COVID-19 public health emergency, as did the Chinese government (11). These guidelines (12,13) stress ensuring the health and safety of trial participants, and suggest alternative measures should be proportionate and based on benefit-risk considerations with adequate documentation. For instance, if the subject is unable to attend the site, then home nursing, contact via phone or telemedicine, location assessment, and other measures may be required to identify adverse events and ensure continuous medical care and oversight for patients. These measures may be helpful to avoid further burden in terms of time and staffing in clinical trials during the COVID-19 pandemic. Moreover, the Chinese medical institutions, such as the clinical trials sites, usually release some specific and detailed working instructions according to national legislation and guidelines. It is crucial to understand whether, and in what ways, the current pandemic situation continues to impact clinical trials in China, which would guide the implementation of ongoing and upcoming clinical trials both in China and internationally. To date, there has been no large sample size study conducted on the impact of the COVID-19 pandemic on clinical trials. As research staff have conducted the trials and experienced the impact firsthand, their assessments are critical to understanding the impact of COVID-19 on trials. In China, research staff generally refers to all team members in clinical trials in sites, including physicians, nurses, technicians, investigators’ assistants—such as clinical research coordinators (CRCs)—and the staff in institutional supervision departments for clinical trials in hospitals—such as personnel in offices of Good Clinical Practice (GCP) and the ethical committee (EC).

CRC is a profession in China which has emerged with the rapid development of clinical trials in recent years. Given the enormous daily outpatient volume of most investigators in China (14), Chinese CRCs often help with most tasks that do not require medical judgment, such as registering patients’, scheduling patient’s computerized tomography (CT) appointments and following-up visits, entering data into case report forms (CRFs), handling trial master files (TMFs), and collaborating with the clinical research associates (CRAs) from sponsors, among other duties (15). After years of development and improvement, the CRC has become an indispensable position in clinical trials (16,17). Numerous well-trained CRCs are active in the front line of clinical research and are also the main force of clinical research teams. Although CRCs are always the unsung heroes of clinical trials, they should have the right to relay the impact of the COVID-19 pandemic on trials. In our survey, CRCs were notably invited as the main participants, and those fulfilling all of the other roles in clinical trials were also invited to provide their feedback and opinions. We aimed to provide an objective, full-scale, and comprehensive view regarding the impact of COVID-19 on clinical trials in China. We present the following article in accordance with the SURGE reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-22-777/rc).

Methods

Development of the questionnaire

This was a non-interventional study. We designed a self-administered questionnaire to capture researchers’ perceptions and attitudes about the impact of the COVID-19 pandemic on clinical trials in China.

The online questionnaire consisted of 4 sections.

In section I, a cover letter explained the purpose of this study and the anonymity and confidentiality of the survey. In section II, items that were relevant to the respondents’ demographics characteristics, including current role, age, gender, and region, were queried.

Section III included 29 items to assess perceptions of the impact of COVID-19 on clinical trials, with scores ranging from 1 (strongly disagree) to 5 (strongly agree), and to compare a period of the past one and a half years to the pre-pandemic period. Each item described a scenario identified from the literature and key informant interviews that was related to one of the following categories:

• Subject enrollment: number of patients or healthy volunteers, informed consent, assessment of eligibility, and investigators’ activities regarding recruitment.
• Patient care: patient engagement, treatment, and retention related to remote follow-up visits; number of safety events; and labs/imaging/testing.
• Study supplies: sufficiency of resources and materials applied to the research, such as investigational products and equipment, as well as their transportation, dispensation, recycling, and destruction.
• Data management: data entry, source document verification and reviewing, and TMFs.
• Quality management: frequency and approaches of monitoring, inspection, and institutional quality control activities.
• Research milestones: willingness to undertake trials, changes in review and approval pathways, research completion status, and the application of telemedicine and other technologies.

In section IV, an optional open question collected respondents’ suggestions on how to ensure the implementation of a good quality clinical trial under the pandemic situation.

Initially, 4 experts from well-known study sites with years of clinical trial experience, including 2 physicians, 1 pharmacist, and 1 administrator, conducted the pilot survey, and we further amended and optimized the questionnaire to achieve high internal consistency reliability [intraclass correlation coefficient (ICC) 0.869 for 29 items in the questionnaire].

Data collection

A convenience sample of research team members was conducted concerning clinical study in China. The respondents were recruited from 272 study sites throughout mainland China, specifically, all tertiary hospitals with GCP qualifications in 121 cities in 30 provinces, autonomous regions, and municipalities (Tibet, Hong Kong, Macao, and Taiwan were not included) between September and October 2021. We identified these study sites from the government database of clinical research institutions (18), which was developed by the National Medical Products Administration (NMPA) in mainland China. The database includes the name, address, contacts, telephone number, and principal investigators of over 1,000 clinical trial institutions. To ensure a sufficient number of responses, we sent the questionnaires only to the eligible and available people at each study site. Due to the different sizes of individual study sites, the number of questionnaires distributed at each study site varied. The initial plan of this survey was to collect 5–20 questionnaires at each study site, with the goal of 2,500 respondents in total. After obtaining permission from the administrative office of the hospitals, 2 well-trained research assistants in each province distributed the online questionnaires to the potential participants through an online survey platform (Survey Star; Changsha Ranxing Science and Technology, Changsha, China). The online voluntary and anonymous questionnaire secured the confidentiality of the participants and did not collect any identity-exposing information of the participants. All participants were informed about the study before accessing the online questionnaire and provided consent before he or she started to complete the questionnaire.

The minimum sample size required for this survey was estimated to be 377, according to the Raosoft sample size calculator (Raosoft Inc., Seattle, WA, USA) using the following formula (19), where n was the sample size required, N the population size, x the confidence interval (CI) which was considered 95%, and e the margin of error which was set to 5%:

n=Nx(N−1)e2+x[1]

Ethical consideration

The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Ethics Review Board of Nanjing Medical University {No. [2021] 103}.

Data analysis and statistics

Data were analyzed using SPSS 24.0 (IBM Corp., Armonk, NY, USA). The multiple imputation approach was used for missing data. The frequencies, means, and standard deviations (SDs) for each item on the questionnaire were computed, and the statistical significance level was P<0.05. Before testing the hypothesis model, we performed a factor analysis to identify and confirm the questionnaire’s structure. Exploratory factor analysis (EFA) was used to determine the structure of the questionnaire, with the index of Kaiser-Meyer-Olkin (KMO) and Bartlett’s test of sphericity. The recommended number of samples for EFA is 5–10 samples per item. Some experts consider even 3 samples per item as adequate, provided that the percent variance is given and the factor loading is over 0.8 (20). The number of factors (different clusters of respondents’ perceptions) was selected by principal component analysis with preferred eigenvalues of >1.00 and varimax rotation with factor loadings >0.40. When appropriate, the Mann-Whitney test was used to compare statistical values between the scores of different factors in different demographic characteristics groups. Cronbach’s alpha statistical index (α) was used to calculate the reliability or internal consistency of the questionnaire.

Results

General results

Of the 2,500 questionnaires distributed, 2,393 were returned and used for analysis by October 31, 2021, representing a response rate of 95.7%. We only retrieved the information from the questionnaires completed by the respondents. Figure 1 illustrates the regional distribution of all counted respondents. The respondents were from 272 study sites nationwide, including 165 (60.7%) from Eastern China. The geographical distribution of respondents was similar to that of Chinese tertiary hospitals. The respondents’ demographics and relevant characteristics are displayed in Table 1. Among 2,393 respondents, 2,180 (91.1%) were female, 2,015 (84.2%) were in the 21 to 30 age group, and 1,469 (61.4%) were from hospitals in eastern China. In addition, 2,147 were CRCs, accounting for 89.7% of the respondents.

Figure 1 Distribution of study sites in provinces of China.

Factor analysis

The KMO value of the questionnaire was 0.955, and the results of Bartlett’s test showed statistical significance (P<0.05). The EFA results showed that the 29 scenarios regarding the impact of COVID-19 on clinical trials could be classified into 4 factors: “subject enrollment” (SE), “patient care” (PC), “study supplies and data management” (S&D), and “research milestones and quality management” (R&Q).

By applying the 5 levels of the Likert scale (21) in our questionnaire, the attitudes of respondents were recorded from “strongly disagree” to “strongly agree” as 1 to 5, respectively, and the median [interquartile range (IQR)] score of each factor presented the general perception of the respondents.

In addition, the cumulative variance contribution rate was 64.93%, indicating that our results expressed by each factor were within an acceptable range. Cronbach’s α coefficient indicated good measurement reliability for all factors (range, 0.82–0.95).

Table 2 lists the preset scenarios of the impact of the COVID-19 pandemic on clinical trials with scales showing respondents’ perceptions and attitudes towards each item by mean score and SD (mean ± SD). The attitudes of respondents were divided into 3 categories: disapproval (scoring 1 and 2), neutrality (scoring 3), and approval (scoring 4 and 5). It was noted that the approval category was dominant for all items except Q15, Q17, and Q18. Most respondents expressed neutral attitudes to these 3 questions, and the attitude of disapproval for each item accounted for more than that of approval. This phenomenon indicates that most respondents did not agree that the pandemic had caused these unfavorable events, including more serious adverse events (SAEs), missed reports of safety events, or any increase of unblinding events in clinical trials. A further comprehensive analysis of all the respondents’ viewpoints is displayed in Figure 2. The positive or negative effects of the impact was judged according to the content of the question item, which described the clinical trial conduct scenario influenced by the COVID-19 pandemic. Colorful columns are used to represent the 6 preset categories in clinical trials, and the height of each column indicates the mean scores of each item, together with its error bar of SD. The 3 negative impact items are plotted in the positive impact quadrant by a similar but lighter color, in addition to the original-colored columns. In summary, our survey confirmed 7 positive and 19 negative effects regarding the conduct of clinical trials during the COVID-19 pandemic, and 3 negative scenarios were not agreeable in general.

Figure 2 Comprehensive analysis of respondents’ viewpoints to the impact of COVID-19 on clinical trials in China. Factors 1–4 are clusters of questionnaire items according to factors analysis. Factor 1: SE; Factor 2: PC; Factor 3: SS & DM; Factor 4: QM & RM. SE, subject enrollment; PC, patient care; SS, study supplies; DM, data management; RM, research milestones; QM, quality management.

Table 3 summarizes the results of analysis comparing the median (IQR) scores of 4 factors between different groups by Mann-Whitney test. The results showed significant differences between males and females in all 4 factors of SE, PC, S&D, and R&Q (respectively, P<0.001, P=0.009, P=0.019, and P=0.024). Moreover, except for the factor PE, all factors showed significant (P<0.05) differences in the median scores among different age groups and respondent groups.

Suggestions proposed

For section IV, 40.2% (962/2,393) of the respondents voluntarily filled in their answers. The suggestions which were provided in high frequency involved the aspects of strengthening communication between sponsors and investigators, and making full use of the internet, namely, implementing remote monitoring and conducting medical oversight by telemedicine. Suggestions also covered many other aspects of a clinical trial, involving more subject’s training on the awareness of COVID-19 prevention and control, simplifying the bureaucratic requirements and processes related to clinical trials such as EC review and approval in an expedited manner, and establishing specialized pathways for clinical trial subjects. Additionally, respondents considered it crucial that the study sites actively and promptly issue their specific guidelines regarding the conduct of clinical trials.

Discussion

Our study showed that over 2,000 research members from study sites covering the east to the west of China accepted the nationwide survey, and the results exhibited a generally negative attitude toward the impact of the COVID-19 pandemic on the conduct of clinical trials, especially regarding subject recruitment and patient care. Consistent with several studies from some other countries (22-27), the negative impact of the pandemic includes aspects such as inevitable delays in follow-up visits, challenges in handling methods of investigational products, and numerous inconveniences caused by the pandemic. Intriguingly, our study indicated that some positive effects of the pandemic on clinical trials were extensively recognized and appreciated, such as the promotion of telemedicine, online meetings, and remote monitoring.

Difficulties in subject enrollment and patient care are multifaceted. One point was patients’ reluctance or a restriction for them to go to hospital during the pandemic. From the perspective of researchers, they were obliged to prioritize the emergent responsibility of treating, preventing, and controlling the COVID-19 pandemic. The reduced allocation time in clinical studies was inevitable, resulting in delayed site-opening meetings, fewer enrolled subjects, and postponed milestones, like EC approval, data review, and database lock. Notably, China has been implementing extraordinary and strict prevention and control policies of COVID-19 since early of 2020, which has presented extra obstacles to the normal implementation of clinical trials. For instance, conventional on-site drug distribution and face-to-face instruction are restricted if either the patient or the medical institution is in a region with a medium-to-high risk of COVID-19. Nevertheless, more than half of the respondents in our survey voted in favor of the routine prevention and control measures, and they believed these measures would actually help clinical trials in the long run. We suppose that this sentiment is associated with the aim of effectively controlling COVID-19 in China and that the public accept the strict policy.

In this study, we investigated the perceptions of all research staff, including varied roles in clinical trials. During the conduct of our survey, many CRCs responded actively, and thus the opinions and attitudes of this new force in Chinese clinical trials were explored extensively for the first time. The main respondents in our study were CRCs (89.7%), who are assigned to work in various departments of hospital when needed. In China, the majority of CRCs are young women (28,29), and our questionnaire showed a similar demographic characteristic. CRCs have become indispensable assistants for investigators and are involved in many aspects of clinical trials, but not the interventional operations. The investigators’ responsibilities are partially the CRCs’ operational responsibilities. However, previous articles have usually focused on researchers, namely, investigator or sub-investigators, and the voices of CRCs have scarcely been heard or analyzed. Our study endeavored to acquire insight into the impact of COVID-19 in China from the first-line research team members, involving CRCs in particular. Compared with other team members in clinical trials, for example, investigators who are predominantly male and usually have completed sufficient years of high-level education, the CRC group showed lower median scores in 3 out of the 4 factors in our survey. In other words, CRCs might be somewhat conservative to pick a distinct choice. We suppose that this phenomenon may be associated with their youth, relatively fewer years of experience, and the assistance feature of work.

Regarding the 3 questioning items (Q15, Q17, and Q18) which inverted from the quadrant of negative impact to the positive one, the results indicated that most respondents did not believe that more SAEs, missed reports of safety events, or any increase of unscheduled unblinding in clinical trials had occurred due to the pandemic. We believe the application of telemedicine and some other techniques may be relevant since they minimize and offset some negative effects of the pandemic. A study conducted in the USA demonstrated that telemedicine improved accuracy and timeliness of safety event reporting since it enabled immediate, direct transmission of patients’ personal health data from home to the investigators, allowing a prompt assessment and follow-up treatment decision (30). In China, WeChat-based telemedicine has been highly recommended for remote follow-up visits during the pandemic (31). Patient e-diary, usually known as e-Pro equipment, is becoming increasingly popular in clinical trials, especially international, multicenter trials (32). In our survey, respondents were generally supportive of the improvement of technology in clinical trials and viewed it as a positive effect produced by the pandemic. Similar to other countries, remote monitoring and medical oversight have been gradually popularized in China (33). We expect that remote trials will be an addition or even an alternative, if needed, to the traditional on-site trials in the future (34-36). Our respondents indicated that high-tech apps are able to increase subjects’ adaptability and protocol compliance in clinical research. Moreover, remote drug distribution and continuous medical oversight of adverse events in trials are available through telemedicine, which reduces hospital visits, decreases the risk of nosocomial infections and dropout rate (37), and eases the stress on medical resources. Although there are some limitations in telemedicine, such as the inability to collect bio-specimens, the difficulty in performing physical examination, and the poor quality of the internet provider network in low-income areas, it is still recommended as a routine element in future clinical trials.

In addition to grading their attitudes to the preset scenarios, some respondents provided suggestions, and the following considerations were offered. First, communication between clinical trial stakeholders should be strengthened, which is crucial during the pandemic, so that problems, if any, can be resolved in a timely manner. Second, guidelines on risk areas should be formulated and updated promptly, and thus the counterplans might be put forward quickly, which may involve, for instance, training people in using personal protective equipment and the handling of patients and staff if they report COVID-19 symptoms and a history of travel to a risky area. Many cancer centers in the USA have implemented relevant policies (38). Third, the resources required for clinical trials should be systematically integrated and allocated, the bureaucratic process related to clinical trials should be simplified, and even specialized pathways should be considered for facilitating the implementation of clinical trials. Fourth, information technologies are favorable, and more equipment, such as remote monitoring systems and wearable sensors (39), is suggested. Fifth, when CRAs cannot conduct on-site monitoring as planned, the staff at study sites should conduct more frequent quality controls to ensure the quality of clinical trials. Finally, increased training and attention should be given to subjects to improve their compliance and stabilize their mood during the pandemic. A study from Italy about cancer care during COVID-19 pointed out that regular contact with patients should be maintained remotely, and that this strategy could also help to relieve patient anxiety and loneliness through psychological interventions (40). The above views were derived from young oncologists in Italy and coincided with the views of respondents in our study.

There are some limitations to our study. First, CRCs constituted the bulk of respondents, and most respondents were young females, possibly indicating selection bias, which might have influenced the results. Our study represents CRCs more than other roles in clinical trials. Differences between varied roles and ages were further compared by rank sum test and discussed. Second, there are limitations based on the observational nature of the study design, as subjects’ recall and response bias are inevitable. Moreover, it is often argued that questionnaire-based surveys are limited in terms of rich and dense descriptive data (41,42). Third, the use of a cross-sectional design rendered us unable to make causal inferences because it did not control for all possible confounding variables, so the differences between groups only indicated the relevance. Fourth, the number of distributed and completed questionnaires differed by site. We had one site that returned several completed questionnaires, while others only sent back one. Our study was conducted on a convenience sampling of research team members of clinical trials, and thus the sample could represent population bias. Furthermore, given the heterogeneity of the center organizations and COVID-19 burden across China, it would be interesting to compare practices between the various regions of China. Our study revealed no significant (P<0.05) difference across the eastern, central, and the western regions in general, and thus investigation on specific regions, such as provinces, will be carried out in future research.

Conclusions

This study explored the impact of the COVID-19 pandemic on clinical trials in China from the perspectives of research team members. The current pandemic situation has indeed had a significant impact on clinical trials, especially in terms of subject recruitment and protocol compliance, but the research team members are still confident and positive about the policies to offset the negative impact. As mentioned in our study, many new technologies and some pragmatic suggestive measures have changed and will continue to change the way in which clinical trials are conducted.

The COVID-19 pandemic remains a major global crisis that will have an everlasting impact on clinical research. People from all walks of life are endeavoring to mitigate the negative impact with joint efforts, and the quality of subsequent clinical trials should be promising.

Acknowledgments

We would like to express our gratitude to all the staff who helped to distribute questionnaires in each province.

Funding: This study was supported by grants from the National Natural Science Foundation of China (No. 72074123), the Soft Science Project of Jiangsu Science and Technology Department (No. BR2020043), and the Science and Technology Planning Project of Suzhou Science and Technology Bureau (No. SR202157).

Footnote

Reporting Checklist: The authors have completed the SURGE reporting checklist. Available at https://atm.amegroups.com/article/view/10.21037/atm-22-777/rc

Data Sharing Statement: Available at https://atm.amegroups.com/article/view/10.21037/atm-22-777/dss

Peer Review File: Available at https://atm.amegroups.com/article/view/10.21037/atm-22-777/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-777/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the Ethics Review Board of Nanjing Medical University {No. [2021]103}. The online voluntary and anonymous questionnaire secured the confidentiality of the participants, and did not collect any identity-exposing information of the participants. All participants were informed about the study before accessing the online questionnaire, and a consent from each participant was provided before he or she started to respond.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Saini KS, Tagliamento M, Lambertini M, et al. Mortality in patients with cancer and coronavirus disease 2019: A systematic review and pooled analysis of 52 studies. Eur J Cancer 2020;139:43-50. [Crossref] [PubMed]
2. Waterhouse DM, Harvey RD, Hurley P, et al. Early Impact of COVID-19 on the Conduct of Oncology Clinical Trials and Long-Term Opportunities for Transformation: Findings From an American Society of Clinical Oncology Survey. JCO Oncol Pract 2020;16:417-21. [Crossref] [PubMed]
3. Poggio F, Tagliamento M, Di Maio M, et al. Assessing the Impact of the COVID-19 Outbreak on the Attitudes and Practice of Italian Oncologists Toward Breast Cancer Care and Related Research Activities. JCO Oncol Pract 2020;16:e1304-14. [Crossref] [PubMed]
4. Asaad M, Habibullah NK, Butler CE. The Impact of COVID-19 on Clinical Trials. Ann Surg 2020;272:e222-3. [Crossref] [PubMed]
5. Upadhaya S, Yu JX, Oliva C, et al. Impact of COVID-19 on oncology clinical trials. Nat Rev Drug Discov 2020;19:376-7. [Crossref] [PubMed]
6. Upadhaya S, Yu JX, Hodge J, et al. COVID-19 impact on oncology clinical trials: a 1-year analysis. Nat Rev Drug Discov 2021;20:415. [Crossref] [PubMed]
7. Li N, Huang HY, Wu DW, et al. Changes in clinical trials of cancer drugs in mainland China over the decade 2009-18: a systematic review. Lancet Oncol 2019;20:e619-26. [Crossref] [PubMed]
8. Lin L, Chen Y, Yan L, et al. Analysis of clinical trials of new drugs in China as of 2019. Drug Discov Today 2020;25:2080-8. [Crossref] [PubMed]
9. Wang C, Tee M, Roy AE, et al. The impact of COVID-19 pandemic on physical and mental health of Asians: A study of seven middle-income countries in Asia. PLoS One 2021;16:e0246824. [Crossref] [PubMed]
10. Wang Z, Chen G, Liu X, et al. How to safely and smoothly resume trials after the COVID-19 outbreak. J Glob Health 2020;10:020310. [Crossref] [PubMed]
11. National Medical Products Administration. Guidance for Drug Clinical Trials in COVID-19 Pandemic (Draft): National Medical Products Administration, 2020. Available online: https://www.nmpa.gov.cn/yaopin/ypggtg/ypqtgg/20200715110101939.html
12. U.S. FDA. FDA guidance on conduct of clinical trials of medical products during COVID-19 pandemic: Guidance for industry, investigators, and institutional review boards. 2020.
13. European Commission. Guidance on the management of clinical trials during the COVID-19 (coronavirus) pandemic. 2020.
14. Li Y, Gong W, Kong X, et al. Factors Associated with Outpatient Satisfaction in Tertiary Hospitals in China: A Systematic Review. Int J Environ Res Public Health 2020;17:7070. [Crossref] [PubMed]
15. Hornung CA, Kerr J, Gluck W, et al. The Competency of Clinical Research Coordinators: The Importance of Education and Experience. Ther Innov Regul Sci 2021;55:1231-8. [Crossref] [PubMed]
16. Zheng J, Chen Y, Shi X. Investigation and Analysis on the Status of Clinical Research Coordinator in Drug Clinical Trials. Journal of Strait Pharmaceutical 2021;33:223-5.
17. Zhou Y, Tang M, Chen YC, et al. Exploring the introduction of clinical research coordinator management model in drug clinical trial institutions. Chinese Journal of Clinical Pharmacy 2017;26:48-50.
18. Drug and medical device clinical trial institution registration platform. Available online: https://beian.cfdi.org.cn/CTMDS/apps/pub/drugPublic.jsp
19. Raosoft sample size calculator. Available online: http://www.raosoft.com/samplesize.html
20. Knapp TR, Brown JK. Ten statistics commandments that almost never should be broken. Res Nurs Health 2014;37:347-51. [Crossref] [PubMed]
21. 5-Point Likert Scale: The Key To Easily Understanding Your Audience. 2020. Available online: https://www.ombea.com/resources/articles/5-point-likert-scale-the-key-to-easily-understanding-your-audience
22. Ali JK, Riches JC. The Impact of the COVID-19 Pandemic on Oncology Care and Clinical Trials. Cancers (Basel) 2021;13:5924. [Crossref] [PubMed]
23. Alqahtani AM, Almazrou SH, Alalweet RM, et al. Impact of COVID-19 on Public Knowledge and Attitudes Toward Participating in Clinical Trials in Saudi Arabia: A Cross-Sectional Study. Int J Gen Med 2021;14:3405-13. [Crossref] [PubMed]
24. Bonell A, Nadjm B, Samateh T, et al. Impact of Personal Cooling on Performance, Comfort and Heat Strain of Healthcare Workers in PPE, a Study From West Africa. Front Public Health 2021;9:712481. [Crossref] [PubMed]
25. Boughey JC, Snyder RA, Kantor O, et al. Impact of the COVID-19 Pandemic on Cancer Clinical Trials. Ann Surg Oncol 2021;28:7311-6. [Crossref] [PubMed]
26. Lara Gongora AB, Werutsky G, Jardim DL, et al. Impact of the COVID-19 Pandemic on Oncology Clinical Research in Latin America (LACOG 0420). JCO Glob Oncol 2021;7:649-58. [Crossref] [PubMed]
27. Marcum M, Kurtzweil N, Vollmer C, et al. COVID-19 pandemic and impact on cancer clinical trials: An academic medical center perspective. Cancer Med 2020;9:6141-6. [Crossref] [PubMed]
28. Zhu W, Zhao Q, Zhu H, et al. The validation of a questionnaire to delineate clinical research coordinator roles in China. Perspect Psychiatr Care 2020;56:629-35. [Crossref] [PubMed]
29. Zhang BY, Zhang P, Xu Y, et al. Analysis on job satisfaction, job burnout condition and turnover intention of clinical research coordinator in five site management organizations. Chinese Journal of New Drugs 2019;28:1237-42.
30. Liu HH, Ezekowitz MD, Columbo M, et al. The future is now: our experience starting a remote clinical trial during the beginning of the COVID-19 pandemic. Trials 2021;22:603. [Crossref] [PubMed]
31. Zhang QL, Huang ST, Xu N, et al. Application of Remote Follow-Up Via the WeChat Platform for Patients who Underwent Congenital Cardiac Surgery During the COVID-19 Epidemic. Braz J Cardiovasc Surg 2021;36:530-4. [PubMed]
32. Inan OT, Tenaerts P, Prindiville SA, et al. Digitizing clinical trials. NPJ Digit Med 2020;3:101. [Crossref] [PubMed]
33. Degtyarev E, Rufibach K, Shentu Y, et al. Assessing the Impact of COVID-19 on the Clinical Trial Objective and Analysis of Oncology Clinical Trials-Application of the Estimand Framework. Stat Biopharm Res 2020;12:427-37. [Crossref] [PubMed]
34. Šendelj R. Information Technology and Information Management in Healthcare. Stud Health Technol Inform 2020;274:139-58. [Crossref] [PubMed]
35. Polhemus AM, Kadhim H, Barnes S, et al. Accelerating Adoption of Patient-Facing Technologies in Clinical Trials: A Pharmaceutical Industry Perspective on Opportunities and Challenges. Ther Innov Regul Sci 2019;53:8-24. [Crossref] [PubMed]
36. Bunnell BE, Sprague G, Qanungo S, et al. An Exploration of Useful Telemedicine-Based Resources for Clinical Research. Telemed J E Health 2020;26:51-65. [Crossref] [PubMed]
37. Alpuim Costa D, Nobre JGG, Fernandes JP, et al. Impact of the COVID-19 Pandemic on Breast Cancer Management in Portugal: A Cross-Sectional Survey-Based Study of Medical Oncologists. Oncol Ther 2022;10:225-40. [Crossref] [PubMed]
38. Desai A, Sachdeva S, Parekh T, et al. COVID-19 and Cancer: Lessons From a Pooled Meta-Analysis. JCO Glob Oncol 2020;6:557-9. [Crossref] [PubMed]
39. Walton MK, Cappelleri JC, Byrom B, et al. Considerations for development of an evidence dossier to support the use of mobile sensor technology for clinical outcome assessments in clinical trials. Contemp Clin Trials 2020;91:105962. [Crossref] [PubMed]
40. Lambertini M, Toss A, Passaro A, et al. Cancer care during the spread of coronavirus disease 2019 (COVID-19) in Italy: young oncologists' perspective. ESMO Open 2020;5:e000759. [Crossref] [PubMed]
41. Baum F. Researching public health: behind the qualitative-quantitative methodological debate. Soc Sci Med 1995;40:459-68. [Crossref] [PubMed]
42. LoBiondo-Wood G, Haber J. Nursing Research: Methods and Critical Appraisal for Evidence-Based Practice. 8th edition. Missouri: Mosby, 2013.