New reporting items and recommendations for randomized trials impacted by COVID-19 and force majeure events: a targeted approach

Introduction

In early 2020, the World Health Organization (WHO) reported the first cases of pneumonia of unknown etiology, caused by a previously unknown coronavirus, in Wuhan, China. In March 2020, the WHO declared the coronavirus disease 19 (COVID-19), previously termed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), outbreak a global pandemic. By the 17^th of May, 2021, the pandemic has led to more than 162 million people being infected and has caused nearly 3.4 million deaths globally (1).

To mitigate the spread of COVID-19, strict social distancing and prolonged lockdown measures were implemented in many countries. During the ongoing pandemic crisis, despite adaptations such as remote data collection and monitoring (2), the willingness and ability to participate in cancer clinical trials have been adversely affected (3,4). Therefore, the conduct of clinical trials may be significantly impacted (5-7).

Moreover, during the ongoing COVID-19 pandemic, healthcare facilities are under intense pressure due to shortages in human resources, hospitalization of COVID-19 patients, and the disruption of the medical and personal protective equipment supply chains (8). This further presents an unprecedented challenge to the conduct of trials (5,9).

Accurate reporting and analyses can improve the reproducibility of published trials impacted by COVID-19. In addition to pandemics like the COVID-19 pandemic, other force majeure events, such as the war in Ukraine are not predictable, can also impact the conduct of trials. The CONSORT reporting guidelines for randomized controlled trials (RCTs) have been instrumental in upholding the integrity of trial reporting (10). Extensions of the CONSORT reporting guidelines have been proposed to cater to specific clinical situations (11). The pandemic has adversely affected the conduct of clinical trials, and the need to add new and modify reporting items to improve the quality of reporting is essential. Without such modifications, inadequate reporting of RCTs impacted by COVID-19 and other force majeure events may complicate the interpretation of study results, thereby negatively impacting future research and patient care. The purpose of this paper is to understand the deficiency for the reporting of randomized trials impacted by COVID-19, propose new or modified reporting items to address such deficiency, and to conduct a small validation on trials partially impacted by COVID-19.

Methods

Our search process is stratified into two parts: part one focused on understanding the number of trials potentially affected by COVID-19, and part two was directed at the effect of COVID-19 on the published articles. In part one, the search was based on the trial information reported in ClinicalTrials.gov to estimate the number of recently opened trials that could be affected by COVID-19. In part two, a comprehensive literature search was performed using PubMed to identify the recently published articles for information relevant to trials potentially affected by COVID-19. The reporting practice is expected to be similar for studies impacted by the pandemic. Therefore, an investigation of articles published recently should reflect the reporting inadequacy. By examining the current reporting practice, inadequately reported items can be highlighted for improvement.

Number of trials potentially impacted by COVID-19

In the search process on ClinicalTrials.gov, we examined phase III clinical trials conducted in all cancer types, cardiovascular diseases, and diabetes that commenced between January 2017 and December 2021. Specifically, only those trials with “Recruiting” status in February of 2021 for trials commenced between 2017 and 2020 and in June of 2022 for trials started in 2021 were included to obtain the number of trials that would be affected by COVID-19 at these two time snapshots for the corresponding years.

The search was conducted using the function “Advanced Search” with the following criteria being selected: (I) condition or disease, (II) status, (III) phase, and (IV) study commencement period of interests. While the second and third variables remained the same throughout the search process as “Recruiting“ and “Phase 3”, the rest varied. The first search was as follows, “Cancer” was selected, and the study commencement period was set from “01/01/2017” to “12/31/2017” to collect the number of cancer trials in the specified period. Similarly, the search for later years “2018”, “2019”, “2020” and “2021” was performed accordingly. The above procedure was replicated for “Cardiovascular diseases” and “Diabetes”. Trials with a recruitment status other than “Recruiting” were excluded from this study.

Reporting items for assessing the impact of COVID-19

We focused on published phase III RCTs in cancers because the results of the former section showed that the number of oncology trials potentially affected by COVID-19 was the highest. Non-original studies, such as meta-analysis or secondary data analysis, were excluded. Articles published between July and December 2020 in JAMA Oncology, Journal of Clinical Oncology, and The Lancet Oncology were examined. Original research articles meeting our eligibility criteria were included for data extraction. Essential study characteristics collected were: type(s) of cancers, number of treatment arms, number of patients enrolled, and the study locations (multinational versus single-country). The criteria for defining whether each reporting item was correctly reported or not are described below. In addition, a summary table is provided in the supplement (Table S1).

Reporting of study schedule

Reporting elements essential for assessing the impact of COVID-19 on the conducted clinical trial were captured. The months and years when the enrollment began and ended, the months and years when the treatment began and ended, and the last month and year of follow-up (usually defined as data cut-off) were extracted to assess whether the trial schedules were well-reported or not. Studies were classified as “Reported” if the data points discussed above were present in the main text. However, those that did not meet the criteria stated above were categorized as “Not reported”.

Reporting of treatment design

The treatment duration was defined as either the number of chemotherapy or targeted treatment cycles delivered, including the length of each cycle or the radiotherapy treatment duration. This variable was classified as “Reported” if it was described as stated above. However, if studies failed to provide the number of cycles or cycle length, the reporting of this variable was classified as “Not reported”. Trials reporting surgery as the only treatment modality would be labeled as “Reported”.

Reporting of study locations

The reporting of study location was another item collected to understand the impact of the pandemic on clinical trials. The reporting of study locations was classified as “Not reported” if the information could not be extracted from the main text. However, if descriptions in the articles specified that study location could be obtained in the study appendix or online materials, the reporting was classified as “Reported in supplementary material”. For those with study locations reported in the main text, they were first classified as “Reported in main text” and were further subclassified into “Region-level”, “Country-Level”, or “Site-level” according to the reported information.

Reporting of treatment-related events

The reporting of the occurrence of treatment-related events was our primary objective. The collected variables included the reasons for treatment discontinuation and treatment delay. The reporting of the reasons for treatment discontinuation was classified as “Reported” if the information was found in the main text. Otherwise, they were classified as “Not reported”. Trials reporting treatment delays termed as “dose delay”, “dose interruption”, “treatment delay”, or “treatment interruption” were labeled as “Reported”. The absence of reporting in either the main text or supplementary material was labeled “Not reported”. Trials reporting the percentage of patients experiencing treatment discontinuation or delay were further classified as “Reported with reasons”, or “Reported without reasons”.

Statistical methods

Descriptive statistics were used to illustrate the number of patients enrolled (the median, Q1, and Q3) and the length of the recruitment period (the mean and standard deviation). The other variables described in the methodology were presented as frequencies and percentages (in two decimal places). A bar plot was used to illustrate the number of trials recruiting participants in oncology, cardiovascular diseases, and diabetes per year between 2017 and 2021.

Results

Study selection and time trend

The total numbers of clinical trials posted between 2017 and 2021 and labeled as “Recruiting” in cancer, cardiovascular diseases, and diabetes were 1,484, 535, and 145 trials, respectively (Figure 1). Comparatively fewer oncology clinical trials commenced in 2020 relative to 2019. This can be the result of either a decrease in registration or a delay in the commencement of oncology clinical trials in 2020.

Figure 1 Numbers of phase III clinical trials with ongoing recruitment as of snapshot 1 (S1) on the 2nd of February 2021 (for trials registered between 2017 and 2020) and snapshot 2 (S2) on 24^th June 2022 (for trials registered in 2021).

The literature search identified 40 potentially relevant articles. After a comprehensive examination, five articles were excluded. The articles excluded from the final analysis were post-hoc analysis (n=2), study updates (n=2), and quality of life (n=1) studies. Thirty-five original articles were included in this study, and five were published in JAMA Oncology, 14 in the Journal of Clinical Oncology, and 16 in The Lancet Oncology (Figure 2).

Figure 2 Process of literature search (Reporting Items for Assessing the Impact of COVID-19).

Characteristics of included articles

The characteristics of the 35 included articles can be found in Table 1. Among these 35 articles, 7 (20%) were genitourinary (GU)-related cancers, followed by breast cancer and gastrointestinal (GI)-related cancers, accounting for 6 (17%) articles each. Thirty (86%) of the studies were designed with two treatment arms, and 5 (14%) had three or more treatment arms. The median number of patients enrolled in the included trials was 513 (interquartile range 301 and 856). Twenty-five (71%) trials reported were multi-center, multinational studies. The remaining 10 (29%) trials were conducted in a single country, primarily in China (5, 14%). The average length of the recruitment period was 3.48 years.

Reporting items for assessing the impact of COVID-19

Table 2 illustrates the evaluation of reporting items for assessing the impact of COVID-19 in included studies. Thirty-three (94%) articles reported the months and years when they commenced and closed patient enrollments, and 20 (57%) articles reported the months and years of data cut-off. In addition, 27 (77%) articles provided information related to treatment lengths, such as the number of and length of chemotherapy cycle and total treatment length of radiotherapy. However, no trials reported the month and year when the first patient received the assigned treatment or the month and year when the last patient received his/her last treatment dose. Regarding the reporting of geographical region or site of recruitment, 8 (23%) trials reported this information in the main text and 17 (49%) in the supplementary materials (Appendix 1), while 10 (29%) did not provide any relevant information. In relation to the treatment effect-related factors, 28 (80%) articles reported the reasons for treatment discontinuation. Nonetheless, only 17 (49%) reported the number or the percentage of patients experiencing treatment delay, with explanations provided in only 9 (26%) trials.

Discussion

Our study demonstrated that more than 1,400 cancer-related clinical trials registered between 2017 and 2021 in the ClinicalTrials.gov database could be affected by COVID-19. In this study, we evaluated the quality of critical reporting items that could potentially be used to evaluate the impact of the COVID-19 pandemic on the conduct and interpretation of oncology clinical trials. Our study findings support the need for new reporting items and recommendations for clinical trials impacted by a pandemic.

For the essential reporting items already included in the current CONSORT guidelines, adequate reporting on these items was low among the existing articles. For example, items related to trial design and trial outcomes were partly reported, indicating the relatively low compliance to the existing guideline. The percentages of reporting the last month and year of follow-up, regions where patients were enrolled, and reasons of treatment delay were under-reported or not reported. Failure to report these items adequately could have consequences on both the reproducibility of clinical trials and the interpretation of results. These consequences are magnified under the impacts of COVID-19. Fifty-one percent of the articles reviewed did not report the proportion of patients experiencing treatment delays. Our results demonstrate that several of these reporting items were inadequately reported. Moreover, we also identified reporting items not currently present in the CONSORT reporting guidelines that are important for trials affected by the COVID-19 pandemic. This was achieved by reviewing existing CONSORT items by section/topic and thoroughly examining recent literature on the impact of the COVID-19 pandemic on trials that are generated by multiple stakeholders (see Table S2) (12-14).

Following the current version of the CONSORT guidelines for trials impacted by COVID-19 would result in inadequate reporting that could hamper result interpretation and reproducibility of clinical trials. Therefore, to ensure that the possible impacts of COVID-19 can be inclusively and clearly described, we propose a set of new reporting items and recommendations (Table 3, with an expanded version integrated with CONSORT in Table S3) for the reporting of clinical trials handled during a pandemic. Table 3 contains the added ten new (Items 1c, 4c, 4d, 5b, 7c, 12c, 13c, 14c, 19b, and 19c) and four modified (items 12b, 14a, 18, and 22) reporting items. Notably, given that sensitivity analysis and missing data analysis were both important for assessing the impact of COVID-19 on the conduct of trials (15), we modified items 12b and 18 to reflect this. In addition, we would also like to highlight that the period of treatment was added to item 14a. As supported by the result in Table 2, this item is currently not routinely reported. However, this can help understand the potential impact of COVID-19 on trial conduct. Lastly, item 22 has been modified so that the authors are reminded to discuss how COVID-19 might have affected their trial interpretation. In particular, they are reminded to discuss the potential bias in interpreting the actual treatment effect.

To further demonstrate how trials may be impacted by COVID-19, a third literature search was conducted in PubMed to validate our proposed new reporting items and recommendations. Original research articles on phase III trials published between January and December 2021 in JAMA Oncology, Journal of Clinical Oncology, and The Lancet Oncology were examined. The literature search identified 81 potentially relevant articles. After excluding articles that did not meet the inclusion criteria that data collection should be completed no earlier than March 2020, 29 articles remained (Figure S1).

These 29 articles were examined using applicable reporting items in our new items and recommendations as these studies were impacted by COVID-19 during their follow-up periods (Table 3). Among these articles, the term “COVID-19” was found in eight articles, only one mentioned COVID-19 in the abstract (item 1c), and two discussed the impacts of COVID-19 in the interpretation section (item 22). In addition, 15 articles mentioned that remote data capture is accepted in their supplementary (item 4d). Two described the modifications on the data cut-off date due to COVID-19 (item 5b) in the main text, and five reported adverse events caused by COVID-19 (item 19b) either in the main text or in the supplement. Among the five articles that reported changes to trial outcomes, only one failed to identify whether it happened before or after the pre-planned interim analysis. The reasons for the delay in treatment (item 13c), including whether they were attributed to COVID-19, were reported in six articles. Twenty-six out of 29 articles indicated that additional analyses were performed, 11 performed sensitivity analysis (item 12b; item 18), and 16 out of 29 articles addressed the issue of missing data (item 12c). However, as described earlier, only eight articles discussed COVID-19 in the text. Therefore, it remains unclear whether these additional analyses were conducted to evaluate the impact of COVID-19. In this brief assessment, six out of seven applicable reporting items that can be assessed were reported in less than 21% of the articles (Table 4). This indicates that the authors reporting trial results impacted by COVID-19 could greatly benefit from following our recommendations for transparent and clear reporting of clinical trial results.

This checklist should be considered as a tool to improve the reproducibility and consistency of reported trials impacted by COVID-19. A diagram has been created to help authors choose whether they should follow our new recommendations (Figure S2). Authors are advised to check whether the new items and recommendations should be followed based on the following statements: (I) a clinical trial site closed to patient recruitment or under lockdown due to COVID-19 and (II) a COVID-19 outbreak occurring at any stage between the enrollment of the first patient and the time of data cut-off at the study location. If the answer is “yes” to either statement, authors are recommended to follow our recommendations for reporting trial findings. In addition, authors may consider using our new recommendations if the clinical trial was affected by a disruption in the drug supply chains due to the COVID-19 pandemic.

While transparent and comprehensive reporting of clinical trials was the primary focus of our study, we suggest suitable statistical analysis methods for clinical trials impacted by the COVID-19 pandemic. In light of the complexities of trial reporting during an ongoing pandemic, as addressed in this study, additional analyses, such as sensitivity analyses, should be performed (16). Regarding missing data, different methods (e.g., complete case analysis, single imputation, and multiple imputations) may be applied. Relevant descriptions and discussions on the utilization of these analytical methods can be found in the articles published by Molenberghs and Kenward (17) as well as O’Kelly and Ratitch (18). In addition, detailed analytical strategies targeting missing data under pandemic were also provided (19). For analysis of pandemic-related missing data and delayed assessments (e.g. scans and laboratory tests), the usage of interval censoring methods is recommended (7), with a more advanced analytical method for interval-censored data proposed in Fu and Simonoff’s study (20). The addition of interim and sensitivity analysis or conducting a final analysis following early termination of trials are described further in the Food and Drug Administration guidelines (15).

Our recommendations may improve the overall quality of trial reporting during a pandemic. Another strength of this study is the applicability of the proposed recommendations to clinical trials, irrespective of the discipline. Furthermore, it is also expected that our work is applicable to not only trials that has been affected by COVID-19, but also other force majeure events, which includes the war in Ukraine and any pandemic that might happen in the future. Notably, we noticed that a relevant article was published recently. The guideline provided by Orkin et al. (21) focused on reporting the modifications to trial protocols and completed trials that were impacted by extenuating circumstances. Existing consort reporting items are judged by the authors based on three options: no change, impact, and mitigating strategy. Unlike the flexibility they provide, our approach is more targeted and provides recommendations focused on ten new and four modified items relevant to trials impacted by the pandemic. As their focus is different, the majority of the new and modified reporting items are unique in our recommendations. Specifically, new or modified reporting items 1c, 4c, 7c, 12b, 13c, 14c, 18, 19b, and 19c have not been covered or mentioned. Therefore, our easy to use checklist is complementary to what is currently available to the best of our knowledge.

Despite the clear strengths of our study, there are some limitations. One limitation is that our study was conducted on articles published in the three top oncology journals. Hence, the observed proportions of inadequate reporting are likely underestimated. However, our recommendations may be more impactful as reporting quality tends to be lower if we consider all journals (10). Another limitation is that we did not assess a large number of trials conducted during the pandemic that are likely to be published in the coming years. However, as mentioned earlier, the reporting practice of recently published studies is expected to be similar to those impacted by COVID-19. Our recommendations aims to act as a preventative measure by reducing reporting inadequacy.

Conclusions

In conclusion, our key findings highlight the need to re-emphasize and refine the set of reporting items of CONSORT for clinical trials conducted during a pandemic or other force majeure events. Therefore, we propose a set of new and modified reporting items for authors to safeguard transparency and enhance the quality of reporting and value of trials impacted by trials impacted by various types of force majeure events.

Acknowledgments

Funding: The research work was partially supported by the National Institutes of Health (Nos. P01CA142538 and R01AG066883 to XW); and University Postgraduate Fellowships of HKU Foundation (to THL).

Footnote

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-2160/coif). THL reports this study partially supported by University Postgraduate Fellowships of HKU Foundation. EEV reports consulting fees and payment/honoraria from AbbVie, AstraZeneca, Beigene, BioNTech, Eli Lilly, EMD Serono, Genentech/Roche, GlaxoSmithKline, Merck and Novatis, outside the submitted work. HP reports personal fees from Genentech, outside the submitted work. The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. World Health Organization. WHO Coronavirus (COVID-19) Dashboard [Internet]. WHO; 2021 [cited 2021 Feb. 25]. Available online: https://covid19.who.int/
2. Guo T, Chen C, Chiang C, et al. Operational Experiences in China and Statistical Issues on the Conduct of Clinical Trials During the COVID-19 Pandemic. Stat Biopharm Res 2020;12:438-42. [Crossref] [PubMed]
3. Constantinou C, Kolokotroni O, Mosquera MC, et al. Developing a holistic contingency plan: Challenges and dilemmas for cancer patients during the COVID-19. Cancer Med 2020;9:6082-92. [Crossref] [PubMed]
4. Koinig KA, Arnold C, Lehmann J, et al. The cancer patient's perspective of COVID-19-induced distress-A cross-sectional study and a longitudinal comparison of HRQOL assessed before and during the pandemic. Cancer Med 2021;10:3928-37. [Crossref] [PubMed]
5. Akacha M, Branson J, Bretz F, et al. Challenges in Assessing the Impact of the COVID-19 Pandemic on the Integrity and Interpretability of Clinical Trials. Stat Biopharm Res 2020;12:419-26. [Crossref] [PubMed]
6. Padala PR, Jendro AM, Padala KP. Conducting Clinical Research During the COVID-19 Pandemic: Investigator and Participant Perspectives. JMIR Public Health Surveill 2020;6:e18887. [Crossref] [PubMed]
7. Meyer RD, Ratitch B, Wolbers M, et al. Statistical Issues and Recommendations for Clinical Trials Conducted During the COVID-19 Pandemic. Stat Biopharm Res 2020;12:399-411. [Crossref] [PubMed]
8. Mirchandani P. Health Care Supply Chains: COVID-19 Challenges and Pressing Actions. Ann Intern Med 2020;173:300-1. [Crossref] [PubMed]
9. Walker RJ, Jackson JL, Asch SM, et al. Mitigating the Impact of COVID-19 on Funded Clinical Research: Crucial Next Steps. J Gen Intern Med 2021;36:518-20. [Crossref] [PubMed]
10. Ghimire S, Kyung E, Lee H, et al. Oncology trial abstracts showed suboptimal improvement in reporting: a comparative before-and-after evaluation using CONSORT for Abstract guidelines. J Clin Epidemiol 2014;67:658-66. [Crossref] [PubMed]
11. Komorowski AS, MacKay HJ, Pezo RC. Quality of adverse event reporting in phase III randomized controlled trials of breast and colorectal cancer: A systematic review. Cancer Med 2020;9:5035-50. [Crossref] [PubMed]
12. Boughey JC, Snyder RA, Kantor O, et al. Impact of the COVID-19 Pandemic on Cancer Clinical Trials. Ann Surg Oncol 2021;28:7311-6. [Crossref] [PubMed]
13. Ali JK, Riches JC. The Impact of the COVID-19 Pandemic on Oncology Care and Clinical Trials. Cancers (Basel) 2021;13:5924. [Crossref] [PubMed]
14. Onesti CE, Tagliamento M, Curigliano G, et al. Expected Medium- and Long-Term Impact of the COVID-19 Outbreak in Oncology. JCO Glob Oncol 2021;7:162-72. [Crossref] [PubMed]
15. US Food and Drug Administration. Statistical Considerations for Clinical Trials During the COVID-19 Public Health Emergency: Guidance for Industry 2020.
16. Moskowitz CS, Panageas KS. Implications for Design and Analyses of Oncology Clinical Trials During the COVID-19 Pandemic. JAMA Oncol 2020;6:1326-7. [Crossref] [PubMed]
17. Molenberghs G, Kenward M. Missing data in clinical studies: John Wiley & Sons; 2007.
18. O'Kelly M, Ratitch B. Clinical trials with missing data: a guide for practitioners: John Wiley & Sons; 2014.
19. Cro S, Morris TP, Kahan BC, et al. A four-step strategy for handling missing outcome data in randomised trials affected by a pandemic. BMC Med Res Methodol 2020;20:208. [Crossref] [PubMed]
20. Fu W, Simonoff JS. Survival trees for interval-censored survival data. Stat Med 2017;36:4831-42. [Crossref] [PubMed]
21. Orkin AM, Gill PJ, Ghersi D, et al. Guidelines for Reporting Trial Protocols and Completed Trials Modified Due to the COVID-19 Pandemic and Other Extenuating Circumstances: The CONSERVE 2021 Statement. JAMA 2021;326:257-65. [Crossref] [PubMed]