Current usage and challenges of master protocols—based on survey results by ASA BIOP oncology methodology working group master protocol sub-team

Introduction

The rapid development of genomic technologies such as next generation sequencing has enabled the use of genomic profiling and biomarkers in drug development (1). The concept of “precision medicine” encourages evaluating investigational treatments matched to patients based on genomic profiling. As a result, prevalence of the corresponding study population can be quite limited. At the same time, increasing number of investigational drugs are evaluated in clinical trials according to the number of original Investigational New Drug applications (IND) by FDA (2), making patient resources for clinical trials more competitive. The traditional drug development paradigm where a single experimental treatment is evaluated in a single disease population has become more expensive and suboptimal in terms of patient resource and development timeline. Master protocol trials that simultaneously evaluate more than one investigational drug and/or more than one disease population within the same overall trial structure (3) have the potential of increasing trial efficiency. To account for varying characteristics and sponsor requirements of the clinical trials, three types of master protocols are often used, classified as basket trials, umbrella trials, or platform trials (4). We use the same definition of basket trials, umbrella trials and platform trials as in (4). Basket trial is defined as a trial that tests a single investigational drug or drug combination in different populations defined by disease stage, histology, number of prior therapies, genetic or other biomarkers, or demographic characteristics. Umbrella trial is defined as a trial that evaluates multiple investigational drugs administered as single drug or as drug combination in a single disease population, where all investigational drugs (or combinations) are enrolled at the same time and with no rolling arm option. A platform trial is a trial that allows flexibility to add new treatment arms in the future, or a hybrid of different disease indications and different treatment or treatment combination in the same trial. Although rising in popularity as summarized in multiple systematic review papers (5,6), practitioners still encounter challenges when designing and implementing trials with master protocol framework (7,8). On behalf of the ASA BIOP Oncology Methods Scientific Working Group Master Protocol Sub-team, the authors of this article have been charted to conduct a survey with the goals of understanding the status of master protocol usage and more importantly the background and stories behind the designs, as well as the challenges and roadblocks that sponsors are facing. This is valuable information for the community to identify and address the roadblocks. We summarize the design and conduct of the survey in ‘Methods’ section and report the survey results in ‘Results’ section. We then further elaborate on the challenges of master protocols in ‘Challenges of master protocols’ section and end with discussions in ‘Discussion’ section. We present the following article/case in accordance with the SURGE reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-21-6139/rc).

Methods

Survey designs and distribution

To keep the survey succinct, three key aspects are included in the survey: (I) current usage of master protocols across different organizations and the clinical phases of the usage of such designs, (II) statistical features including usage of randomization, adaptations, adjusting for multiplicity, and inclusion of non-concurrent control, and (III) challenges of designing, implementing and engaging stakeholders for such trials. A total of 19 questions are included in the survey. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

The survey was set up on SurveyPlatnet.com and was active between April and May 2021 for participants to fill out. To avoid potential duplication of answering the survey from the same function area within the same organization, the ASA BIOP Master Protocol sub-team members reached out to their contacts on a pre-determined list with 37 organizations (Appendix 1) covering major pharmaceutical companies, biotechnology companies, academic centers, and non-profit organizations. The list of distributed companies includes 19 out of the top 20 pharmaceutical companies ranked in year 2020 (9). While attempting to reach out to as many top 21–50 pharmaceutical companies and academia/non-profit organizations as the working group’s network covered, we also added a few biotechnology companies to increase diversity so organizations vary in size, number of employees and the coverage of therapeutical areas. One response from one organization was generally collected to avoid duplication. While considering the contact person from each organization, we targeted those who oversee a larger portion of the biometrics team, if not the entire biometrics organization, thus may have a bigger overview of the trials conducted in the organization. We advised the contact person to coordinate with other leaders within their organization for the survey results. For example, for large global pharmaceutical companies, this contact person may be the statistical head of the company, or the head of one therapeutic area in statistics, and would have scheduled a meeting with other therapeutical areas (separating oncology vs. the other therapeutic areas)/regional statistical heads to get the details of the master protocols conducted within the company that he does not oversee. After that, the contact person would consolidate the information and answer the survey. With that said, a few exceptions were made when it comes to large global companies where oncology and non-oncology organizations are less connected, one representative from oncology and another from non-oncology TAs participated in the survey. With this setting, results presented in this report are typically based on the number of responders as denominators. No incentives were given to the sub-team members who reached out or the participants who answered the survey. One reminder was sent by the sub-team members before the deadline for the participants they reached out to, in order to complete the survey with high participation rate.

Results

A total of 39 responses were received from the 37 organizations contacted with a few large organizations providing more than one response. Almost all participants that were reached out to filled out the survey, with response rate higher than 90%. Among the participants, 32 (82%) were from pharmaceutical companies/biotechnology companies, and 7 (18%) were from academic centers/non-profit organizations (Figure 1). All data from all respondents are accounted for in the survey. Count and percentage are reported for each question. Depending on the context of the question, the denominator could be different and the percentages for all categories may or may not add up to 100%. Missing data are listed as a separate category whenever applicable.

Figure 1 Organizations of participation.

Thirty-one (79%) respondents answered that they have had at least one master protocol trial either in planning or in implementation in their organization (Figure 2).

Figure 2 Use of master protocols.

Among the 8 respondents who indicated they did not have master protocols planned, 6 (75%) did not encounter the need for having master protocols in their drug development endeavor; 1 (25%) considered the master protocol during design phase and decided it was too complex for consideration; and 1 (25%) respondent did not specify why no master protocol was conducted.

Table 1 summarizes various characteristics of the master protocols trials being planned separated by pharmaceutical companies and academic centers. Majority of master protocol trials are in oncology, but not limited to only oncology. All three types of master protocol trials are planned, and majority of the master protocols are exploratory (phase I, phase II or phase I/II). Contrary to the FDA guidance on master protocol (3), most of the master protocols do not use independent data monitoring committee (IDMC) for monitoring mid-trial data which may be due to the fact that most of the master protocols reported were used in exploratory fashion.

During the research and discussions of master protocols in the ASA oncology Working Group, we found out that different countries and regulatory agencies may have different requirements and opinions about the designs, conduct for master protocol trials, especially if the purpose of the trial is for regulatory registration. Hence, we put in a question specifically on this aspect. Among the 31 respondents with master protocol trials planned or conducted, 8 had at least one master protocol with registrational intent, 50% (4 out of 8) indicated that the regulatory feedback is generally supportive and consistent, while 25% (2 out of 8) indicated they received inconsistent feedback from different regulatory agencies where some were supportive, and some were less supportive. The remaining did not provide details in write-in.

Statistical features of the master protocol studies are also collected. Despite some high-level considerations in the FDA guidance for master protocols, there are still a lot of debates on major statistical considerations for master protocol trials, such as, if randomized trials are preferred for proof-of-concept umbrella trials, if trial level family-wise type I error should be controlled, and if non-concurrent control could be used for the primary analysis. The purpose of this survey is to get an idea what the common practice is for master protocols in 2021 and would see if the trend would change in a few years. Table 2 summarized the study design features related to randomization and use of control. According to the survey results, more studies are using concurrent control in their primary analyses. This is in contrary to FDA’s guidance for confirmatory trials for COVID-19 master protocols (10). Reason for this is that majority of the master protocol trials surveyed are exploratory trials, which have the needs to optimize the possible use of the trial data to increase efficiency.

A total of 22 (71%) indicated they have adaptive design features included in the master protocols. The most used adaptive design feature is treatment arm or population adding or dropping that accounted for 50% among the 22 responses with adaptive designs features, followed by response adaptive randomization (5 out of 22, 23%), sample size re-estimation (3 out of 22, 14%), and sequential monitoring (3 out of 22, 14%) (Figure 3).

Figure 3 Adaptive design features in master protocol trials.

Most of the organizations did not attempt to control the study level family-wise type I error rate (FWER) in master protocol trials (Figure 4A). When FWER control is required in the study, simulations (rather than analytical) approach is the more frequently used approach (Figure 4B).

Figure 4 (A) Whether family-wise type I error control is implemented. (B) How family-wise type I error control is demonstrated.

Challenges of master protocols

The survey also collected the challenges of planning and conducting master protocols as an open question, where respondents can write in free text. We share in this section the challenges provided by survey participants along with perspectives and considerations from our scientific working group members in the context of some recent publications.

In summary, survey respondents reported statistical, operational, and regulatory obstacles to the design and conduct of master protocols.

Three statistical challenges were reported, including (I) difficulty in evaluating the statistical properties of the master protocol by clinical trial simulation, (II) lack of guidance on the use of non-concurrent control patients, and (III) lack of guidance on multiplicity control. Master protocols can be complex statistically, and clinical trial simulations are needed to evaluate statistical properties such as type I error rates, power, and bias in estimation. Survey respondents reported that one barrier is a lack of readily available software to perform these tasks. For basket trials, there are commercially available software packages such as FACTS (Berry Consultants) and EAST Bayes (Cytel). Additionally, there are several R packages available, including “basket-package” and “bhmbasket”. Meyer et al. (11) reviewed the currently available software for platform trial simulations, some of which are commercially available and some of which are available in R. Even with available software, customization are needed to allow more flexibility. As to lacking guidance on the usage of non-concurrent control patients and multiplicity control, various authors including representatives from regulatory agencies have presented their reviews in the past years (4,8,10). Project SignifiCanT (Statistics in Cancer Trials) (12), jointly hosted by ASA Biopharmaceutical Statistical Methods in Oncology Scientific Working Group and LUNGevity Foundation in coordination with Oncology Center of Excellence in FDA, has also clarified the joint opinions from global regulatory and industry/academia on these two topics (13,14). With the fast evolvement of these research areas, we believe more clear guidance and consensus will emerge in near future.

The operational concerns reported by survey participants focused generally on the increased operational complexity of the master protocol approach versus the traditional stand-alone trial. Some detailed aspects were included in the reports: (I) survey participants indicated longer time required to plan and initiate a master protocol trial comparing to the traditional one, to address issues over the development of the master protocol documents themselves, and the scale and complexity of the trial processes including data collection and reporting mechanisms. The master protocol trial will have multiple study documents including the master protocol itself and the appendices that define each sub-study. Generally, details that govern all sub-studies go into the master protocol while intervention specific details go into the appendices with the intervention specific details overruling the general master protocol specifications if there is a conflict. However, with the complexity of master protocol trials, these general rules may not always be straightforward under all circumstances. (II) Building and maintaining a trial infrastructure that can support the fast paced and constantly changing nature of these trials presented another operational challenge. Master protocol trials may be in a constant state of change with interim analyses to update randomization probabilities across arms and/or to add/drop sub-studies, especially in an open-ended platform trial. Given the inevitability of constant change for these types of master protocol trials, it is critical to plan and budget accordingly for ongoing adjustments. This may include changes to study documents such as the protocols, appendices, and informed consents, that may require multiple committees’ approval, as well as changes in the therapies under consideration and corresponding changes in clinical suppliers and drug administration procedures (8). (III) From patient enrollment perspective, many master protocol trials have patient eligibility to the trial and assignment of therapies based on specific biomarkers. The availability of validated biomarker assays and their timely processing could be a roadblock to enrollment and assignment of therapy especially if patient eligibility is determined by multiple assays (8). It is recommended that an optimal drug/technology development program include prospective planning on simultaneous evaluation of assay technology and drug compounds (15). (IV) Perhaps one of the biggest operational challenges, however, is to have a system that ensures the timeliness and quality of data, which may directly impact the conduct of the trial. All adaptive trials need to be able to have good quality data in real time to conduct interim analyses and this needs to be coupled with trial processes that allow fast data reviews, decision making, and implementation of recommended actions. To maintain high quality data in real time, one may consider centralized monitoring approaches, as outlined in International Council Harmonisation (ICH) Guidelines (16), that use pre-defined key risk indicators (e.g., trial conduct, data integrity, safety concerns) together with advanced analytic methods to identify real-time potential issues from a large volume of data.

Finally, some survey participants reported a lack of confidence that regulatory authorities would accept a master protocol trial particularly with registrational intent. Several participants indicated that, in general, EU regulatory is in more favor of less complex proposals than other agencies. Other participants indicated that a high volume of questions were received from global regulatory agencies but felt the questions ultimately helped to improve the trial designs. Some participants also reported that sometimes inconsistent regulatory feedbacks were received from global regulatory agencies that different agencies may have different levels of acceptance of master protocol trials as confirmatory trials intended for market applications. Although this is generally the case for complex innovative trials, it certainly poses challenges for master protocol designs to be widely used, especially for registration trials. Encouragingly, there has been more public collaborative efforts made in recent years among the regulatory agencies globally, which indicates the awareness of the challenges sponsors are facing due to disparate requests from global regulatory agencies. One example in the master protocol trial space is that FDA and EMA worked together to publish a guidance document for developing drugs for rare pediatric Gaucher disease where a controlled multi-arm, multi-company clinical trial is encouraged to facilitate the development of multiple drugs (17,18). Another example of global regulatory agencies working collaboratively is the Project Orbis (19), where a framework is established to facilitate concurrent submission/review of oncology drugs and simultaneous decisions among multiple regulatory agencies. While these great initiatives indicate an exciting trend of global regulatory agencies working collaboratively to provide guidance and review to sponsors, a more consolidated framework to review complex innovative designs such as master protocol design features before the trial initiation would be extremely desirable for sponsors who may see extended regulatory review cycles and sometimes contradictory views from various agencies as a roadblock to such designs.

Discussion

In this paper, the survey on the current usage and challenges of the master protocol trials that was conducted between April to May 2021 by ASA BIOP oncology methodology working group, master protocol sub-team was reported. The results of the survey were able to address the objectives of better understanding the use of the master protocols and the roadblocks. As far as we are aware of, this is the first master protocol survey conducted in the United States that is not literature review based.

With limited resources in the working group, the survey was designed with some limitations. First, as mentioned in the design section, to avoid duplicate reporting and ensure survey quality, the decision was made to hand pick organizations and representatives from each organization to complete the survey, which is the strength of the survey. However, the survey did not drill down to the level of each individual trial but rather focused on the overall usage and experience from each participant. Therefore, the survey results are based on the number of responding organizations rather than on the number of individual trials. In other words, a participant from a large organization may respond to the survey with multiple master protocol trials in mind while another participant from a small organization may respond to the survey with only one trial experience in mind. While organizational-based survey results are reasonably acceptable for some of the questions, for other questions such as master protocol trials usage by phases (phase I, II, III etc. or the specific statistical features included in the master protocol, it may be more desirable to obtain an answer at the trial level. Secondly, the survey distribution is dependent on the working group members’ network coverage. Therefore, a certain level of bias may exist as a higher percentage of pharmaceutical companies were contacted than academic, government, or other non-profit organizations, which may or may not reflect the distribution in the community who may conduct master protocol trials. However, since the major organizations that may conduct master protocol trials are mostly covered, we consider the biases due to the organizational coverage to be limited. Thirdly, we thought through options to avoid duplicate reporting within each organization. However, for cross-industry collaborative trials, such as I-SPY2, duplicate reporting may not be avoidable. A potential solution for this limitation is to request names or national clinical trials (NCT) numbers for each master protocol trial in the survey. However, for trials yet to be registered or openly published, there will be confidentiality considerations for the participants to reveal such details. Therefore, with the considerations of relieving confidentiality concerns from the survey participants, the sub-team decided not to include the question of requesting trial identifications.

The master protocol framework is a highly evolving topic. The applications are increasing exponentially in the past two decades (7). Part of the intention of our survey is to serve as a landmark on the usage of master protocols. We plan to repeat a survey in a few years and observe the applications, practical considerations, design features and the associated shift in mindset, if any, in this field. Note as the first survey in the time series, we intended to focus on a relatively small number of organizations and gradually increase the number of organizations to be surveyed in the follow-up survey when more experiences are gained, and more resources are available. Similar approach was taken by the DIA Innovative Design Scientific Working Group on their series of surveys over the years (20-22). Meanwhile, the survey we currently conducted mostly focused on statistical considerations. However, a successful application of master protocol framework requires not only statistical excellence but also awareness and effective collaborations from all related functions including clinical, regulatory, operations and many more. With that in mind, the plan is to conduct another survey focusing on multidisciplinary aspects to identify roadblocks and challenges more broadly in the near future.

Although the focus of this paper is the current usage and challenges of the master protocol framework, we would like to highlight the encouraging side of the story: the global regulatory agencies are increasingly supportive of master protocol framework as a special type of complex innovative design (CID). FDA recently published “Final Guidance COVID-19: Master Protocols Evaluating Drugs and Biological Products for Treatment or Prevention”, and finalized the guidance “Master Protocols: Efficient Clinical Trials Design Strategies to Expedite Development of Oncology Drugs and Biologics” after their initial draft guidance in 2018 (3,10). They also include master protocols as part of CID and encourage sponsors to submit the protocol via the CID process to obtain more frequent and in-depth feedback from them (23). Upon sponsor’s agreement, some of the designs may be shared publicly with the community so the learning and lessons can be shared broadly (24,25). In some areas of rare diseases (e.g., Duchenne Muscular Dystrophy), regulatory agencies have made public comments in support of the efforts that pharmaceutical community works together on one single protocol to bring effective treatment to patients, which smooths many roadblocks in implementing such an innovative design (25).

Conclusions

We are in an exciting era in the history of drug development, where innovations are not bounded by traditions or precedent, where multiple stakeholders working together with one end goal in mind: most efficiently bring the innovative drug to patients who need it. With many innovations such as master protocol framework, our community will conquer the seemingly impossible, with no limits.

Acknowledgments

We thank Hong Tian, Yevgen Tymofyeyev, Xiaoming Li, Suman Sen, Yingwen Dong, Stefan Englert, Freda Cooner, Melissa E Spann, Richard Volk, Olga Marchenko, Venkat Sethuraman, Kevin Chartier, Zhenming Shun, Robert Wan, Sammi Tang, Mattew Guo, Ruitao Lin, Jared Foster, Ji Yuan, Yang Song, Kyle Wathen, Robert Beckman, Amy Stark, Megan Othus, Xiaofei Wang, Sammi (Rui) Tang, Linda (Zhiping) Sun, Janice Grechko, and Yuhwen Soo, Claudine Isaacs, Inna Perevozskaya and Jingshan Zhang (the order the names is random) for helping collect the data for this manuscript.

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Annals of Translational Medicine for the series “Challenges in Clinical Trials”. The article has undergone external peer review.

Reporting Checklist: The authors have completed the SURGE reporting checklist. Available at https://atm.amegroups.com/article/view/10.21037/atm-21-6139/rc

Conflicts of Interest: The authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-21-6139/coif). The series “Challenges in Clinical Trials” was commissioned by the editorial office without any funding or sponsorship. JY served as the unpaid Guest Editor of the series. PB is an employee of Pfizer and stock is part of compensation plan. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Dugger SA, Platt A, Goldstein DB. Drug development in the era of precision medicine. Nat Rev Drug Discov 2018;17:183-96. [Crossref] [PubMed]
2. Number of Original Investigational New Drug (IND) applications received in the quarter. Available online: https://www.accessdata.fda.gov/scripts/fdatrack/view/track.cfm?program=cber&status=public&id=CBER-All-IND-and-IDEs-recieved-and-actions&fy=2019
3. Master Protocols: Efficient Clinical Trial Design Strategies to Expedite Development of Oncology Drugs and Biologics. Guidance for Industry. U.S. Department of Health and Human Services. Food and Drug Administration; March 2022.
4. Lu CC, Li XN, Broglio K, et al. Practical Considerations and Recommendations for Master Protocol Framework: Basket, Umbrella and Platform Trials. Ther Innov Regul Sci 2021;55:1145-54. [Crossref] [PubMed]
5. Meyer EL, Mesenbrink P, Dunger-Baldauf C, et al. The Evolution of Master Protocol Clinical Trial Designs: A Systematic Literature Review. Clin Ther 2020;42:1330-60. [Crossref] [PubMed]
6. Park JJH, Siden E, Zoratti MJ, et al. Systematic review of basket trials, umbrella trials, and platform trials: a landscape analysis of master protocols. Trials 2019;20:572. [Crossref] [PubMed]
7. Burd A, Schilsky RL, Byrd JC, et al. Challenges and approaches to implementing master/basket trials in oncology. Blood Adv 2019;3:2237-43. [Crossref] [PubMed]
8. Cecchini M, Rubin EH, Blumenthal GM, et al. Challenges with Novel Clinical Trial Designs: Master Protocols. Clin Cancer Res 2019;25:2049-57. [Crossref] [PubMed]
9. Pharm Exec's Top 50 Companies 2020. Available online: https://www.pharmexec.com/view/pharm-execs-top-50-companies-2020
10. FDA final Guidance COVID-19: Master Protocols Evaluating Drugs and Biological Products for Treatment or Prevention. Food and Drug Administration; May 2021.
11. Meyer EL, Mesenbrink P, Mielke T, et al. Systematic review of available software for multi-arm multi-stage and platform clinical trial design. Trials 2021;22:183. [Crossref] [PubMed]
12. Sridhara R, Marchenko O, Jiang Q, et al. Type I error considerations in master protocols with common control in oncology trials: report of an American statistical association biopharmaceutical section open forum discussion. Stat Biopharm Res 2021:1-4.
13. Sridhara R, Marchenko O, Jiang Q, et al. Use of Non-concurrent Common Control in Master Protocols in Oncology Trials: Report of an American Statistical Association Biopharmaceutical Section Open Forum Discussion. Stat Biopharm Res 2021:1-6.
14. Project Significant: Statistics in Cancer Trials Promoting collaboration in design and analysis of cancer clinical trials. Available online: https://www.fda.gov/about-fda/oncology-center-excellence/project-significant-statistics-cancer-trials.
15. Renfro LA, Sargent DJ. Statistical controversies in clinical research: basket trials, umbrella trials, and other master protocols: a review and examples. Ann Oncol 2017;28:34-43. [Crossref] [PubMed]
16. ICH (2018). E6(R2) Good Clinical Practice: Integrated Addendum to ICH E6(R1). Available online: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/e6r2-good-clinical-practice-integrated-addendum-ich-e6r1
17. Paediatric Gaucher Disease. A strategic collaborative approach from EMA and FDA, European Medicines Agency. Science Medicines Health, 2017.
18. Pediatric Rare Diseases — A Collaborative Approach for Drug Development Using Gaucher Disease as a Model. Guidance for Industry, U.S. Department of Health and Human Services. Food and Drug Administration; December 2017.
19. Project Orbis: Strengthening International Collaboration for Oncology Product Reviews, Faster Patient Access to Innovative Therapies. Available online: https://www.fda.gov/news-events/fda-voices/project-orbis-strengthening-international-collaboration-oncology-product-reviews-faster-patient
20. Quinlan J, Gaydos B, Maca J, et al. Barriers and opportunities for implementation of adaptive designs in pharmaceutical product development. Clin Trials 2010;7:167-73. [Crossref] [PubMed]
21. Morgan CC, Huyck S, Jenkins M, et al. Adaptive Design: Results of 2012 Survey on Perception and Use. Ther Innov Regul Sci 2014;48:473-81. [Crossref] [PubMed]
22. Hartford A, Thomann M, Chen X, et al. Adaptive Designs: Results of 2016 Survey on Perception and Use. Ther Innov Regul Sci 2020;54:42-54. [Crossref] [PubMed]
23. Interacting with the FDA on Complex Innovative Trial Designs for Drugs and Biological Products. Food and Drug Administration; December 2020.
24. CID Case Study: Master Protocol to Study Chronic Pain. Available online: https://www.fda.gov/media/155403/downloadParent Project Muscular Dystrophy. Available online: https://www.parentprojectmd.org/research/current-research/our-strategy-impact/master-protocol/
25. Parent Project Muscular Dystrophy. Available online: https://www.parentprojectmd.org/research/current-research/our-strategy-impact/master-protocol/