Establishment of a novel risk score for in-hospital mortality in adult sepsis patients

Introduction

Sepsis is a common critical disease in the emergency department and intensive care unit (ICU) (1). Due to its rapid deterioration and insidious onset, sepsis has high rates of misdiagnosis, mortality, and disability (2). Despite improvements to diagnostic methods and treatment measures, the incidence and mortality of sepsis remain high, and thus it is presently a prominent global health challenge (3).

To predict the outcomes of critically ill patients quickly and accurately, numerous scoring standards have been proposed. Currently, the most commonly used scoring standards in clinical practice include the Acute Physiology and Chronic Health Evaluation II scoring system (APACHE II) and the Modified Early Warning Score (MEWS).

The APACHE II score includes acute physiological indicators, chronic health status, and age, which can be used to quantify the condition of critically ill patients and objectively evaluate the possibility of organ failure and death in these patients (4). However, the APACHE II score is not an assessment of the prognosis of patients with sepsis, and is time-consuming to calculate, with poor clinical operability.

The MEWS is a simple and rapid scoring system that focuses on the patient’s heart rate (HR), systolic blood pressure (SBP), respiratory rate (RR), body temperature, and conscious state (5). However, it lacks confirmation of large sample studies, and cannot fully reflect the pathophysiologic process of sepsis. The results of a multicenter prospective study in Italy showed that MEWS could not effectively predict sepsis in death (6). Similarly, a meta-analysis of 4,298 patients from six studies also suggested that the MEWS was not very effective in predicting sepsis mortality (7).

Following the sepsis-3 criteria update, no scoring systems fit the new diagnosis well and can accurately and easily predict the in-hospital mortality of adult patients with sepsis. The present study was conducted to identify the prognostic factors for in-hospital mortality in adult sepsis patients, in order to develop a simpler and more accurate scoring system, and to evaluate its efficacy compared with that of APACHE II and MEWS. We present the following article in accordance with the TRIPOD reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-21-2900/rc).

Methods

Study design

This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by ethics committee of Ruijin Hospital (No. 2021-59) and informed consent was taken from all the patients.

A total of 1,860 patients diagnosed with sepsis, bacteriaemia, septicemia, septic shock, or infectious multiple organ dysfunction syndromes, who were admitted to the Ruijin Hospital, Shanghai Jiao Tong University School of Medicine (a 1,900-bed, university-affiliated, tertiary hospital in Shanghai, China), were screened from January 1st, 2015, to December 31st, 2019. After excluding 18 patients younger than 18 years of age, 486 patients who did not meet the sepsis-3 criteria, and 21 patients with missing important data, 1,335 patients were included in this study. The enrolled patients were randomly divided into a modeling group (n=801) and a validation group (n=534), at a 3:2 ratio. The study population selection and research process flowchart were shown in Figure 1. Diagnosis of sepsis was based on the sepsis-3 definition and sequential organ failure assessment (SOFA) score (3).

Figure 1 Study population and research process flowchart. APACHE II, Acute Physiology and Chronic Health Evaluation II scoring system; MEWS, Modified Early Warning Score; ROC, receiver operating characteristic.

Data collection

Demographic, clinical, and laboratory data were retrieved from an electronic hospital database, including age, gender, comorbidity, immunosuppressor use, admission department, timing of sepsis diagnosis, length of stay (LOS), hospitalization expenses, site of infection, blood culture positivity, neutropenia, central venous catheterization, invasive intervention or surgery, vasopressin, mechanical ventilation, hemodialysis, urine output, Glasgow coma scale (GCS) score, maximum temperature (Tmax), minimum temperature (Tmin), HR, RR, SBP, diastolic blood pressure (DBP), mean artery pressure (MAP), partial pressure of carbon dioxide (PaCO₂), partial pressure of oxygen (PaO₂), C-reactive protein (CRP), procalcitonin (PCT), lactic acid (Lac), white blood cell (WBC), neutrophil (NE), hemoglobin (Hb), platelet (PLT), hematocrit (HCT), creatinine (Cr), total bilirubin (TBil), serum potassium ions (K⁺), cardiac troponin I (cTnI), aspartate aminotransferase (AST), alanine aminotransferase (ALT), γ-glutamyl transpeptidase (γ-GT), activated partial thromboplastin time (APTT), prothrombin time (PT), and D-dimer (D-D).

Vital signs and laboratory data were measured within 24 hours of sepsis diagnosis. The primary endpoint was in-hospital death. LOS was defined as the number of days between the date of admission to the ward and the date of discharge or death. Neutropenia was defined an absolute NE count of less than 1,500 per microliter (1,500/mL).

Statistical analysis

The collected data were statistically analyzed using SPSS (ver.22.0; Chicago, IL, USA) and MedCalc (ver.19.7; Ostend, Belgium) software. Data were tested for normal distribution using the Kolmogorov-Smirnov and Shapiro-Wilk tests. Continuous variables were expressed as mean ± standard deviation (SD) in cases of normally distributed data, or as median (first quartile, third quartile) in cases of non-normally distributed data. Analysis was carried out using the Student’s t-test or the Mann-Whitney U test, as appropriate. Categorical variables were recorded, frequency percentages were calculated, and the χ² test was used for these analyses. Multivariable logistic regression analyses were performed to identify independent prognostic factors for mortality using variables with P values <0.1 in the univariable analyses. Also, the Hosmer-Lemeshow goodness-of-fit test was used for evaluating calibration. Discrimination was assessed using the area under the receiver operating characteristic (ROC) curve (AUROC). DeLong test was used to compare the differences in the AUROC. Odds ratios (ORs) and 95% confidence intervals (CIs) were calculated. P values <0.05 were considered statistically significant.

Results

Baseline characteristics

A total of 1,335 adult inpatients with sepsis were included, and 34.6% were women. The average age was 61.31 (range, 18–107) years. These patients were divided into two groups: survivors (n=785, 58.8%) and non-survivors (n=550, 41.2%).

Compared with the survivor group, the non-survivor group was older (59.34±17.74 vs. 64.13±17.22 years, P<0.001), had a lower body mass index (BMI) (24.01±12.96 vs. 22.74±4.43 kg/m², P=0.028), higher Charlson comorbidity index (CCI) {1 [0–3] vs. 2 [1–4], P<0.001}, greater use of immunosuppressor (5.4% vs. 9.3%, P=0.006), and shorter LOS {27 [15–45] vs. 19 [8–40] days, P<0.001}. In the non-survivor group, the respiratory system was the most common infection site (P<0.001), and more patients were diagnosed with sepsis after 48 hours compared to the survivor group (26.5% vs. 42.5%, P<0.001). No significant differences were observed between the two groups in gender and admission department (P=0.436, P=0.319, relatively). The results are shown in Table 1.

Comparison of baseline characteristics between the modeling and validation groups

As shown in Table 2, there were no statistical differences in the various variables between the modeling and validation groups.

Establishment of the in-hospital mortality risk factors in the modeling group in adult sepsis inpatients

The clinical data in the modeling group were compared between the survival and non-survivor groups (Table 3). Variables including age, CCI, immunosuppressor use, sepsis diagnosis after 48 h, neutropenia, central venous catheterization, mechanical ventilation, vasopressin, invasive intervention or surgery, DBP, RR, Tmax, HR, GCS, PaCO₂, Lac, Hb, PLT, HCT, AST, K⁺, cTnI, APTT, PT, and D-D showed statistical significance between the two groups (P<0.05). Further multivariate logistic regression analysis indicated that age, CCI, central venous catheterization, mechanical ventilation, vasopressin, RR, HR, GCS, PLT, HCT, AST, and APTT were independent risk factors for in-hospital death (shown in Table 4).

Establishment of the risk score for in-hospital mortality in adult sepsis inpatients

We converted the continuous variables into categorical variables according to the cutoff point of continuous variables and clinical experience as follows: age: 18–69, 70–79, ≥80 years; CCI: 0–1, 2–4, ≥5; RR: <16/min, ≥16/min; HR: <125/min, ≥125/min; GCS: 13–15, 9–12, 3–8; PLT: <20×10⁹/L, ≥20×10⁹/L; HCT: <0.284%, ≥0.284%; AST: ≤36 U/L, >36 U/L; and APTT: <43.0 s, ≥43.0 s. We further analyzed the new variables with multivariate logistic regression (shown in Table 5), and the corresponding integrals of various OR values were endowed according to the principle of round. Each patient’s score was calculated. As shown in Table 6, 0 points for age 18–69 years, 2 points for age 70–79 years, 4 points for age ≥80 years; 0 points for CCI 0–1, 2 points for CCI 2–4, 4 points for CCI ≥5; 3 points for without central venous catheterization; 2 points for mechanical ventilation; 6 points for vasopressin; 0 points for RR <16/min, 3 points for RR ≥16/min; 0 points for HR <125/min, 3 points for HR ≥125/min; 0 points for GCS 13–15, 2 points for GCS 9–12, 6 points for GCS 3–8; 2 points for PLT <20×10⁹/L, 0 points for PLT ≥20×10⁹/L; 2 points for HCT <0.284%, 0 points for HCT ≥0.284%; 0 points for AST ≤36 U/L, 2 points for AST >36 U/L; 0 points for APTT <43.0 s, 2 points for APTT ≥43.0 s, with a total score of 39 points (Table 6).

Comparison between the novel risk score, APACHE II, and MEWS in predicting in-hospital mortality in adult sepsis inpatients

The predictive value of the novel risk score for predicting in-hospital mortality in the modeling group (AUROC =0.857, 95% CI: 0.831–0.881) was superior to the APACHE II score (AUROC =0.767, 95% CI: 0.736–0.796) and the MEWS (AUROC =0.805, 95% CI: 0.776–0.832) (Figure 2A). The DeLong test showed that Z was 4.433 and 2.980, respectively, with a P value of <0.001 and 0.003, respectively. After validating the novel risk score in the validation group, similar outcomes were observed. The novel risk score (AUROC =0.819, 95% CI: 0.783–0.851) was superior to the APACHE II score (AUROC =0.756, 95% CI: 0.717–0.792) and the MEWS (AUROC =0.772, 95% CI: 0.734–0.807) (Figure 2B). The DeLong test showed that Z was 2.323 and 1.923, respectively, with a P value of 0.02 and 0.05, respectively.

Figure 2 Validation of the novel risk score. (A) Comparison of the AUROCs between the novel risk score, the APACHE II score, and the MEWS in the modeling group. (B) Comparison of the AUROCs between the novel risk score, the APACHE II score, and the MEWS in the validation group. (C) Comparison of the AUROCs of the novel risk score between the modeling and validation groups. APACHE II, Acute Physiology and Chronic Health Evaluation scoring system; MEWS, Modified Early Warning Score; AUROC, area under the receiver operating characteristic curve.

We also compared the predictive value of the novel risk score between the modeling and validation groups (shown in Figure 2C). The DeLong test showed that Z was 1.675, with a P value of 0.09, which indicated that the novel risk score had good repeatability.

The risk levels according to the novel risk score

The mortality risk score was divided into four quartiles: low risk level [0–9], lower medium risk level [10–13], higher medium risk level [14–17], and high risk level [18–39]. The actual mortality rates of the modeling and validation groups were 9.1% and 10.9%, 27.0% and 20.9%, 61.4% and 48.7%, and 87.3% and 78.7%, respectively (Figure 3). In both the modeling and validation groups, there were significant differences in actual mortality among the different risk level groups (P<0.001).

Figure 3 In-hospital morality of the different levels of the novel risk score between the modeling and validation groups.

Discussion

In this study, the in-hospital mortality rate of sepsis reached 41.2%, which was higher than other domestic studies (8,9). We hypothesized that this might be due to the fact that some community-acquired sepsis patients were first diagnosed and treated in the emergency department, and after appropriate treatment, their SOFA score was less than 2 points when admitted to the wards. Thus, these patients were excluded in our study.

Considering that the diagnostic criteria for sepsis have been controversial, and the fact that clinicians’ understanding of sepsis has not been updated, we found that misdiagnosis of sepsis is quite common in internal medicine and surgical wards. We hereby call for strengthen of sepsis awareness.

In this study, we found that a total of 442 patients (33.1%) developed sepsis 48 hours after admission, and we speculated that this might have been hospital-acquired sepsis. Among them, 234 people died, with a mortality rate as high as 52.9%, which accounted for 42.5% of all deaths. This mortality rate is higher than that of community-acquired sepsis, and thus, we call for more attention to be paid to hospital-acquired sepsis, those patients needed further monitoring and treatment in ICU unit.

Through univariable and multivariable logistic regression analysis, twelve variables were finally screened, including two basic situations (age and CCI), three clinical interventions (mechanical ventilation, vasopressin, and central venous catheterization), three vital signs (HR, RR, and GCS), and four laboratory results (PLT, HCT, AST, and APTT). Among these, central venous catheterization, GCS, PLT, and HCT were independent protective factors for in-hospital death in adult sepsis patients, while the remaining eight variables were independent risk factors.

PLTs have received increasing attention due to their role in the pathophysiology of infectious diseases, inflammatory responses, and immunity. In 1976, Bone et al. explained the relationship between thrombocytopenia and sepsis (10). PLTs can release cytokines, recruit WBCs, and interact with bacteria and endothelial cells to promote microthrombosis. These mechanisms are adaptive and protective in the context of a locally-controlled infection, but become unregulated during sepsis, leading to impaired organ function (11). Thrombocytopenia in patients with sepsis is not only an indicator of poor prognosis for sepsis; sepsis-associated thrombocytopenia itself may increase the risk of death. Both Claushuis et al. and Thiery-Antier et al. found that PLT count <50×10⁹/L is indicative of poor prognosis of patients with sepsis (12,13). Similarly, PLTs were found in our study to be a protective factor for nosocomial death in adults with sepsis. A study by Vandijck et al. showed that critically ill patients with a PLT count <20×10⁹/L had a mortality rate of 77.8% (14).

APTT is also one of the most commonly used indicators to reflect the coagulation activity of endogenous coagulation systems in clinical practice. Niederwanger et al. and Benediktsson et al. showed that the extension of APTT is a risk factor for the death of children with sepsis and ICU patients (15,16).

Central venous catheterization has many advantages in critically ill patients; it provides secure and lasting vascular access for drug infusion and parenteral nutrition. However, an indwelling central venous catheter may increase the risk of iatrogenic infection, which might cause bloodstream infections (17), and thus, the use of a central venous catheter is a double-edged sword. In this study, central venous catheterization was found to be an independent protective factor. We speculated that it provides vascular access for sepsis patients, and facilitates early fluid resuscitation and parenteral nutrition. We compared the proportion of central venous catheterization in different wards, among which, the surgical ward (56.8%) and the ICU ward (54.1%) were significantly higher than the medical ward (21.9%).

The variables in the novel risk score are all easily obtained. The four laboratory tests (HCT, PLT, APTT, and AST) are routine test items for hospitalized patients. Compared with the other risk scores such as MEWS and APACHE II, our score does not require some indicators that are difficult to obtain, such as PaO₂ and FiO₂, and does not require indicators that are not routinely detected outside of ICU wards, such as Lac and PCT.

This study had several limitations that should be noted. Firstly, it was a retrospective study and was conducted at a single center in China, and no external validation was carried out in this study, which limits the generalization of its findings. Secondly, the sepsis-3 criteria were updated in 2016, so there may have been some missing cases during 2015–2016. Thirdly, some community-acquired sepsis patients were first diagnosed and treated in the emergency department, and after appropriate treatment, their SOFA score might have been less than 2 points when admitted to wards. Thus, these patients were excluded in our study.

Conclusions

In conclusion, age, central vein catheterization, mechanical ventilation, vasopressin, CCI, RR, HR, GCS, PLT, HCT, AST, and APTT were found to be independent risk factors for in-hospital death in adult sepsis patients. We developed and validated a novel risk score, which better predicts mortality than the APACHE II score and the MEWS.

Acknowledgments

Funding: This work was supported by the Program for Outstanding Medical Academic Leader, National Nature Science Foundation of China (Nos. 81772107, 81772040); the Scientific and Technological Innovation Act Program of Science and Technology Commission of Shanghai Municipality (No. 18411950900); the Clinical Research Plan of Shanghai Hospital Development Center (No. SHDC2020CR1028B); and the Program of Shanghai Jiao Tong University School of Medicine (No. DLY201803).

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://atm.amegroups.com/article/view/10.21037/atm-21-2900/rc

Data Sharing Statement: Available at https://atm.amegroups.com/article/view/10.21037/atm-21-2900/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-21-2900/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. This study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by ethics committee of Ruijin Hospital (No. 2021-59) and informed consent was taken from all the patients.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Seymour CW, Rosengart MR. Septic shock: advances in diagnosis and treatment. JAMA 2015;314:708-17. Erratum in: JAMA 2015;314:1404. [Crossref] [PubMed]
2. Winters BD, Eberlein M, Leung J, et al. Long-term mortality and quality of life in sepsis: a systematic review. Crit Care Med 2010;38:1276-83. [Crossref] [PubMed]
3. Shankar-Hari M, Phillips GS, Levy ML, et al. Developing a new definition and assessing new clinical criteria for septic shock: for the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA 2016;315:775-87. [Crossref] [PubMed]
4. Knaus WA, Zimmerman JE, Wagner DP, et al. APACHE-acute physiology and chronic health evaluation: a physiologically based classification system. Crit Care Med 1981;9:591-7. [Crossref] [PubMed]
5. Subbe CP, Kruger M, Rutherford P, et al. Validation of a modified Early Warning Score in medical admissions. QJM 2001;94:521-6. [Crossref] [PubMed]
6. Tirotta D, Gambacorta M, La Regina M, et al. Evaluation of the threshold value for the modified early warning score (MEWS) in medical septic patients: a secondary analysis of an Italian multicentric prospective cohort (SNOOPII study). QJM 2017;110:369-73. [Crossref] [PubMed]
7. Hamilton F, Arnold D, Baird A, et al. Early Warning Scores do not accurately predict mortality in sepsis: a meta-analysis and systematic review of the literature. J Infect 2018;76:241-8. [Crossref] [PubMed]
8. Xie J, Wang H, Kang Y, et al. The epidemiology of sepsis in Chinese ICUs: a national cross-sectional survey. Crit Care Med 2020;48:e209-18. [Crossref] [PubMed]
9. Weng L, Zeng XY, Yin P, et al. Sepsis-related mortality in China: a descriptive analysis. Intensive Care Med 2018;44:1071-80. [Crossref] [PubMed]
10. Bone RC, Francis PB, Pierce AK. Intravascular coagulation associated with the adult respiratory distress syndrome. Am J Med 1976;61:585-9. [Crossref] [PubMed]
11. Singer M. The role of mitochondrial dysfunction in sepsis-induced multi-organ failure. Virulence 2014;5:66-72. [Crossref] [PubMed]
12. Claushuis TA, van Vught LA, Scicluna BP, et al. Thrombocytopenia is associated with a dysregulated host response in critically ill sepsis patients. Blood 2016;127:3062-72. [Crossref] [PubMed]
13. Thiery-Antier N, Binquet C, Vinault S, et al. Is thrombocytopenia an early prognostic marker in septic shock? Crit Care Med 2016;44:764-72. [Crossref] [PubMed]
14. Vandijck DM, Blot SI, De Waele JJ, et al. Thrombocytopenia and outcome in critically ill patients with bloodstream infection. Heart Lung 2010;39:21-6. [Crossref] [PubMed]
15. Niederwanger C, Bachler M, Hell T, et al. Inflammatory and coagulatory parameters linked to survival in critically ill children with sepsis. Ann Intensive Care 2018;8:111. [Crossref] [PubMed]
16. Benediktsson S, Frigyesi A, Kander T. Routine coagulation tests on ICU admission are associated with mortality in sepsis: an observational study. Acta Anaesthesiol Scand 2017;61:790-6. [Crossref] [PubMed]
17. Fahy B, Sockrider M. Central Venous Catheter. Am J Respir Crit Care Med 2019;199:21-2. [Crossref] [PubMed]