How much clinical evidence is enough: regulation of medical devices?

In a recent publication, Marcus and colleagues questioned the adequacy of the amount of high level clinical evidence obtained under the current legal framework for U.S. regulation of medical devices (1). Based on a cross-sectional literature review covering the years 2000 through 2004, they identified clinical research publications for 218 unique medical devices of which 99 (45%) were cleared or approved for marketing in the U.S. The clinical studies associated with the 99 devices were mostly case series or level 4 evidence. In addition to several critical methodological flaws of the study, the authors fail to fully appreciate the current legal framework under which medical devices are regulated in the U.S (2,3).

The methods used in this study have critical flaws that undermine their intent to determine the level of clinical evidence used in medical device regulatory decision-making. For example, the search terms used by the authors to identify clinical studies were limited to only those consistent with early feasibility studies, such as “first clinical”, “early human”, “initial experience”, and “phase 1”. Such studies are by their very nature case series. However, the authors did not seek to identify pivotal clinical studies that may have been conducted to support FDA approval. Moreover, device manufacturers selectively publish study results, particularly if they are negative, or they may significantly delay publication. The absence of published clinical studies does not mean the absence of clinical data. FDA provides publicly available summaries of the premarket approval and clearance decisions it has made, which, particularly for high-risk devices, describes the evidence the agency relied on in its decision making. Therefore, a survey of FDA’s medical device databases (http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/Databases/default.htm) would provide a more accurate and complete estimation of the amount of clinical data obtained pre-market. Of note, the article also limited its search to early studies conducted over a decade ago, which would not necessarily reflect current practices.

It is not surprising that the authors found a higher rate of clearance/approval of devices where industry rather than academia was the sponsor of an early feasibility study. The academicians’ primary motivation may have been publication rather than bringing a device to market. Moreover, industry generally has greater resources to generate the full set of evidence necessary to support, and the regulatory expertise to obtain marketing authorization and potentially reimbursement through FDA’s sister agency, CMS. The flaws of this study significantly underestimate the amount of clinical data obtained in the development of truly new devices. Medical device types range from the very simple (dental floss or tongue depressors) to the highly complex (programmable pacemakers or heart lung machines). Medical devices also vary significantly in their technology such as medical application software, diagnostic ultrasonography, inflatable penile prosthetics, and next generation sequencing machines.

The U.S. standard for marketing a medical device is reasonable assurance of safety and effectiveness (RASE). The wide range of products described above has led to the development of a risk-based approach to regulation. It should be obvious that the amount of information needed to determine RASE for a new dental floss would be significantly different from that of a new cardiac pacemaker. In addition, new lower to moderate risk (some class I and most class II) devices are reviewed prior to marketing under the 510(k) pathway, whereby RASE is established through demonstration of substantial equivalence to an existing marketed device (predicate) that may already have been found to be RASE. Little or no clinical evidence may be needed to demonstrate substantial equivalence. This is analogous to generic drugs that are found to be bioequivalent to a marketed reference product and thus do not normally require clinical evidence to assure safety and effectiveness. Clinical studies are often required for innovative moderate-risk devices under the de novo classification process (4). Non-clinical testing for devices plays a critical role in their evaluation as it offers an opportunity to “test to failure” and evaluate devices under worst case conditions. As a result, in some cases, clinical testing may not be necessary—and in most others—the clinical evaluation is just a part of the full review.

In contrast, higher-risk devices (class III) require pre-market approval prior to marketing and this often entails extensive clinical testing starting with initial feasibility studies and culminating with the pivotal clinical studies needed to demonstrate RASE. Although randomized controlled trials may be required to accomplish this, there are times that, just as with drugs, regulatory decision-making may be based on the literature or study designs that employ non-concurrent controls (5).

The legal framework underpinning the regulation of medical devices is designed to strike a balance between the desire to bring innovative technology to the U.S. public as rapidly as possible, while at the same time assuring that these medical devices are of high quality and safe and effective for their intended use. Regulators may prefer to have full knowledge about a product before allowing it to be marketed, but this is unrealistic and must be balanced by the significant patient need for life-improving and life-saving technology. Thus, post-marketing studies may be required at the time of approval, and all products are continuously monitored for safety (6).

Acknowledgements

None.

Footnote

Provenance: This is a Guest Commentary commissioned by Section Editor Jianrong Zhang, MD (Department of Thoracic Surgery, First Affiliated Hospital of Guangzhou Medical University, Guangzhou Institute of Respiratory Disease, Guangzhou, China).

Conflicts of Interest: The authors have no conflicts of interest to declare.

Comment on: Marcus HJ, Payne CJ, Hughes-Hallett A, et al. Regulatory approval of new medical devices: cross sectional study. BMJ 2016;353:i2587.

References

1. Marcus HJ, Payne CJ, Hughes-Hallett A, et al. Regulatory approval of new medical devices: cross sectional study. BMJ 2016;353:i2587. [Crossref] [PubMed]
2. Jarow JP, Baxley JH. Medical devices: US medical device regulation. Urol Oncol 2015;33:128-32. [Crossref] [PubMed]
3. Maisel WH. Medical device regulation: an introduction for the practicing physician. Ann Intern Med 2004;140:296-302. [Crossref] [PubMed]
4. FDA. De Novo Classification Process (Evaluation of Automatic Class III Designation). Available online: http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/UCM273903.pdf
5. FDA. The Least Burdensome Provisions of the FDA Modernization Act of 1997: Concept and Principles; Final Guidance for FDA and Industry. Available online: http://www.fda.gov/downloads/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm085999.pdf
6. Shuren J, Califf RM. Need for a National Evaluation System for Health Technology. JAMA 2016. [Epub ahead of print]. [Crossref] [PubMed]