Big data to the rescue of systemic inflammatory response syndrome: is electronic data mining the way of the future?

The recognition and accurate diagnosis of sepsis continues to stifle clinicians and researchers alike, as evidenced in part by the recently proposed and ever-evolving clinical definitions (1). Based on available data, global estimates of the burden of sepsis are at 31.5 million cases annually with mortality rates around 20% (2,3). Despite decades of research, clinically efficacious therapies remain elusive. To aid the efforts of researchers and clinicians, the ACCP/SCCM consensus conference in 1992 proposed a set of standard definitions, which have since come to be the cornerstone of clinical sepsis research and influenced clinical practice—enter the systemic inflammatory response syndrome (SIRS) criteria (4).

As SIRS gained popularity for clinical trial inclusion criteria, it was realized that despite their sensitivity, a decided lack of specificity significantly limited their clinical utility (5). The 2001 International Sepsis Definition Conference, however, did not find significant evidence to change the definition of sepsis. Although recognizing the limitations of SIRS criteria, they recommended a set of more clinically useful parameters that might help classify those patients the physician thought “looked septic” but did not meet the formal criteria (6). Despite these changes, subsequent research found that SIRS criteria were not very helpful for risk stratification when these patients presented to the emergency department (ED) (7) and furthermore that they might miss up to one in eight severe sepsis patients, bringing into question even the greatest strength of these criteria, their sensitivity (8). The Third International Consensus Definition for Sepsis and Septic Shock suggests reducing emphasis on SIRS and focusing instead on end-organ dysfunction to identify and prognosticate the outcomes of sepsis patients (1). With that background in mind, what utility do the SIRS criteria hold?

One limitation that the studies of the SIRS criteria have nearly universally suffered from is their reliance on a set of data points from a single point in time. Among a highly dynamic set of variables, such as vital signs, for instance, the assumption that this single set of data points accurately represents the underlying pathology may be inherently flawed. While in many ways 20th century research methodologies forced this means of assessment, the explosion in clinical data available for both research and clinical purposes perhaps calls for a reassessment of the utility of SIRS using a far richer data set. In the July edition of Critical Care Medicine, Lindner et al. further explore the developing field of using modern computational data mining capabilities with their newly derived “SIRS descriptors” to predict and diagnose sepsis in the ICU population (9). The authors must be commended on two novel approaches. First, their approach centered on dynamically collected vital signs in contradistinction to the “spot check evaluations” at various predetermined time intervals in previous studies using SIRS criteria. Second, post-traumatic patients often exhibit SIRS criteria without infection, or “sterile SIRS,” simply as a result of the traumatic injury itself, significantly limiting the application of SIRS criteria in clinical practice. Thus, their chosen study population of a polytrauma cohort makes for a very interesting study.

In their cross-sectional model, the number of SIRS criteria exhibited by each patient for every minute of ICU stay for all their patients was calculated and served as the independent variable of interest. The authors then performed a cohort study where they studied the polytrauma cohort from ICU admission to the outcomes of sepsis diagnosis, death or discharge from the ICU. They showed that in terms of sepsis prediction, compared to “conventional SIRS”, the SIRS predictor “λ” (defined as number of SIRS criteria satisfied each minute with an average value of 1.72) was superior to a single spot measurement, which has face validity and is not particularly surprising, though important to document. From a clinical utility standpoint, perhaps the most interesting and useful finding of the study is that the change in λ (or number of predictors per minute) rises (∆) in the 24 hours leading up to sepsis diagnosis in those patients meeting the study outcomes, but not the controls. These data suggest the evolution of vital signs and number of SIRS criteria per minute may serve as an early indicator of sepsis, and prove to be more useful than the single set of criteria that have proven so unhelpful previously.

While this study provides an interesting perspective on the utility of dynamically collected and calculated SIRS-criteria based descriptors, and the power of automated data mining beyond the capability of any single provider, there remain concerns regarding the approach before these data can be deployed. As with many studies in this field, the lack of a strong gold standard weighs heavily. The outcome of “sepsis” was determined using initiation of “sepsis-specific antibacterial treatment” and “sepsis diagnosis time” was defined as the time of “the first order of the antibacterial”. As clinicians were not blind to vital signs (though pragmatically blinded to λ and ∆), the diagnosis of sepsis might be a self-fulfilling prophecy. Changes in vital signs or laboratory results could easily have prompted a consideration of sepsis diagnosis, leading to antibiotic orders. This act in and of itself led to the diagnosis of sepsis based on the outcome of the study. Therefore, an analysis of the change in SIRS-criteria for these patients to predict and diagnose sepsis may not be the most optimal model to test the validity of the algorithm. In our opinion, the modality of diagnosis used in the study reduces the enthusiasm for the study results. Secondly, their cases (sepsis) in their 5.5-year study period, accounted for only 85 out of >11,000 patients which both seems unusually low and limits inference and generalization.

However, regardless of the methodological issues mentioned above, this paper adds important evidence to the growing body of literature harnessing the power of the electronic health record (EHR) to aid in patient management and quality improvement, especially for time-sensitive disease processes. EHRs are becoming integrated with more and more health care systems and the potential of utilizing the vast amount of minute by minute data to get a more complete picture of the dynamic nature of a patient’s illness cannot be overstated. As the authors note, “electronic surveillance algorithms” have been evaluated for other critical diseases like lung and kidney injury, and this approach has also been investigated in ED sepsis patients (10). Many hospitals are already starting to use EHR data, specifically triage and dynamic vital sign measurements in EDs and ICUs to alert them to potential sepsis patients especially in the current environment of sepsis quality improvement. Using computerized algorithms to extend this static measure to a dynamic concept during a patient’s ED, ICU and ward stay may assist in presenting clinicians with a more holistic picture of disease progression and may serve as an additional tool to heighten a clinician’s awareness of possible sepsis. However, further formalized study of these processes are needed, as more data is not necessarily better data, and issues such as specificity, alarm fatigue, and the adverse consequences of over testing and overtreatment must be weighed against potential benefits.

It is possible to envision even more complex algorithms that harness even more information about the patient from the EHR—past medical history, previous infections, antibiotic resistance patterns, other comorbidities and present more robust, simple-to-use data regarding prognosis and diagnosis in the future. Institution specific dynamic, machine-learning algorithms that consider local practice idiosyncrasies and seasonal variation may even have a role, though such approaches are still in their infancy (11). Big data may change the way we understand and use traditional medical definitions and provide us the opportunity to not only study but intervene at an earlier course in the natural history of the disease. This study shines light on the power of the technology that already exists. The current challenge is to reliably harness the vast quantity of available data in a clinically meaningful way to impact patient care for the better.

Acknowledgements

None.

Footnote

Conflicts of Interest: The authors have no conflicts of interest to declare.

References

1. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:801-10. [Crossref] [PubMed]
2. Fleischmann C, Scherag A, Adhikari NK, et al. Assessment of Global Incidence and Mortality of Hospital-treated Sepsis. Current Estimates and Limitations. Am J Respir Crit Care Med 2016;193:259-72. [Crossref] [PubMed]
3. Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med 2001;29:1303-10. [Crossref] [PubMed]
4. Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992;101:1644-55. [Crossref] [PubMed]
5. Vincent JL. Dear SIRS, I'm sorry to say that I don't like you.. Crit Care Med 1997;25:372-4. [Crossref] [PubMed]
6. Levy MM, Fink MP, Marshall JC, et al. 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med 2003;31:1250-6. [Crossref] [PubMed]
7. Shapiro N, Howell MD, Bates DW, et al. The association of sepsis syndrome and organ dysfunction with mortality in emergency department patients with suspected infection. Ann Emerg Med 2006;48:583-90, 590.e1.
8. Kaukonen KM, Bailey M, Pilcher D, et al. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med 2015;372:1629-38. [Crossref] [PubMed]
9. Lindner HA, Balaban Ü, Sturm T, et al. An Algorithm for Systemic Inflammatory Response Syndrome Criteria-Based Prediction of Sepsis in a Polytrauma Cohort. Crit Care Med 2016;44:2199-207. [Crossref] [PubMed]
10. Nelson JL, Smith BL, Jared JD, et al. Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis. Ann Emerg Med 2011;57:500-4. [Crossref] [PubMed]
11. Taylor RA, Pare JR, Venkatesh AK, et al. Prediction of In-hospital Mortality in Emergency Department Patients With Sepsis: A Local Big Data-Driven, Machine Learning Approach. Acad Emerg Med 2016;23:269-78. [Crossref] [PubMed]