Challenging orthodoxy in critical care trial design: physiological responsiveness

Research involving critically ill adults poses unique challenges in addition to the usual difficulties involved in conducting quality, replicable scientific research. The critical care research community has responded admirably to these challenges by rigorously conducting numerous large and often multicenter randomized controlled trials (RCTs) of putative therapies for critical illnesses. Yet in spite of two decades of work, few incontrovertibly efficacious therapies have resulted from this herculean effort (1). The reasons for this are unsettled, but several categories of problems have emerged. The first problem is that of “positive” trials that cannot be replicated (2-15). Since the clinical trial is in essence a diagnostic test of a hypothesis (16), these non-replicable studies represent “false positives.” False positive trials are due to type I errors which are increased by selection of a conventional and lax statistical significance threshold (e.g., α=0.05) (17,18), bias in the study at any stage of design, conduct, analysis and reporting (19), and fraud (20). Recently, the center for open science collaboration demonstrated that the majority of 100 “positive” psychological research studies could not be replicated, suggesting a false positive rate of 63% in that field (21). The generalizability of this result to medicine is uncertain, but the problems of non-replicability and false positives are not.