A prediction model with measured sentiment scores for the risk of in-hospital mortality in acute pancreatitis: a retrospective cohort study

Introduction

Acute pancreatitis (AP) is an inflammatory disease of the pancreas and is the cause of substantial mortality and morbidity worldwide (1). In the general population, the incidence of AP is 34 per 100,000 people annually and continues to increase (2). As one of the most frequent gastrointestinal causes for hospital admissions, AP leads to $9.3 billion in costs to the healthcare system in the United States each year (3,4). Evidence shows that approximately 20–30% of AP patients can go on to develop severe acute pancreatitis (SAP), which has high mortality and morbidity because of the development of extra-pancreatic and pancreatic necrosis, subsequent infection, and multiple organ failure (5,6). Despite a decreased overall mortality due to improvements in the care of critically ill patients and in accurate and prompt diagnosis, the mortality and long-term sequelae of AP are still considerable (7,8). To implement more effective management, it is of great importance for clinicians to accurately assess the severity and prognosis of AP patients in a timely manner.

In the last few years, scoring systems, such as sequential organ failure assessment (SOFA) and Ranson scores, and laboratory indicators, such as glucose-to-lymphocyte ratio, have been used to assess the prognosis of AP (9-12). However, the predictive performance of the single laboratory indicator is likely to be affected by its fluctuation in accuracy. In addition, despite involvement of about 10 variables, SOFA and Ranson scores both need to be dynamically recorded and their application is limited in early prediction (13). Therefore, it is essential to establish a predictive model that has better accuracy when assessing the prognosis of AP.

Previous studies showed that clinicians were able to predict the mortality in the intensive care unit (ICU), and their notes were valuable to the health status of patients (14-16). The sentiments in clinical notes can reflect the attitude or impression of clinicians to patients, which can be measured through sentiment analysis. These sentiments are found to change with time, clinical characteristics, and outcomes, and are correlated with readmission and mortality (17). In contrast to structured data, the use of unstructured data such as nursing notes showed greater potential to predict mortality (15). To the best of our knowledge, the value of nursing notes in predicting in-hospital mortality of AP is unclear. Herein, we developed a predictive model combined with sentiment scores in nursing notes for in-hospital mortality in AP patients and validated its performance in the prediction of in-hospital mortality. We present the following article in accordance with the TRIPOD reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-22-1613/rc).

Methods

Study design and population

The data used in this retrospective cohort study were accessed from Medical Information Mart for Intensive Care III (MIMIC-III), a large, freely available, public database. MIMIC-III encompasses deidentified health-associated data related to 53,423 hospital admissions for the patients aged ≥16 years who stayed in critical care units at a large tertiary care hospital (Beth Israel Deaconess Medical Center, BIDMC) between 2001 and 2012 (18). The project was initiated by collaborating research groups and researchers from the Massachusetts Institute of Technology (MIT) Laboratory for Computational Physiology and was approved by the institutional review boards of the BIDMC and MIT, with a waiver of informed consent. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Data from patients diagnosed with AP were collected from the MIMIC-III through the computer code of International Classification of Diseases. AP was diagnosed based on the following conditions: (I) abdominal pain related to AP; (II) imaging evidence of AP through computed tomography (CT) scanning and/or ultrasonography; and (III) at least 3-fold increase of lipase and/or amylase levels compared with the normal threshold (13). Patients aged <18 years and those without clinical notes were excluded.

Study variables

Sentiments in the clinical notes were assessed through sentiment analysis, which is an approach of classifying or quantifying the subjective properties of the written text (19). For each individual clinical note, sentiment polarity scores and sentiment subjectivity scores were assigned. In this study, Python’s TextBlob library was used to analyze the sentiment in clinical notes via natural language processing and text analysis (20). The AFINN sentiment lexicon from the polarity scale of −1 (most negative) to 1 (most positive) was used. In addition, subjectivity analysis for each individual clinical note was performed using Python’s TextBlob library and was labeled, with a range of 0 (objective) to 1 (subjective). Higher scores represented more positive and more subjective sentiments, respectively.

Covariates

The health-related data were all extracted from the MIMIC-III, including demographics (age, gender, marital status, ethnicity, etc.), vital signs (temperature, respiratory rate, heart rate and blood pressure), laboratory test indicators [oxygen saturation (SpO₂), oxygen partial pressure (PO₂), partial pressure of carbon dioxide (PCO₂), white blood cells (WBC), red blood cells (RBC), platelets (PLT), RDW, international normalized ratio (INR), mean corpuscular volume (MCV), glucose, hematocrit, hemoglobin, total bilirubin (TBIL), aspartate aminotransferase (AST), alanine aminotransferase (ALT), creatinine, blood urea nitrogen (BUN), albumin, etc.], fluid balance (sodium, potassium, phosphate, calcium, pH value, etc.), first and last care unit [coronary care unit (CCU), cardiac surgery recovery unit (CSRU), medical/surgical/trauma or surgical intensive care unit (MICU/SICU/TSICU)], ICU length of stay (LOS), hospital LOS, medical history [chronic obstructive pulmonary disease (COPD), lung cancer, liver cirrhosis, etc.], subjective mean, subjective minimum, polarity mean and polarity minimum. Moreover, SOFA score and simplified acute physiology score-II (SAPS-II) were used to evaluate the severity of AP according to the MIMIC-III data.

Outcomes

The in-hospital mortality of AP patients was the outcome in this cohort. The follow-up period was the entire hospital stay. Follow-up was terminated if the patient died while hospitalized. A total of 631 patients were included in the study for analysis. Of these, 88 cases died during hospitalization.

Statistical analysis

In this study, a normality test for measurement data was performed with the Kolmogorov-Smirnov test. The measurement data with a normal distribution were compared by t-test and described as the mean ± standard deviation (x¯±s), while those with an abnormal distribution were expressed as the median and interquartile ranges [M (Q1, Q3)] and compared with the Mann-Whitney U rank-sum test. Categorical variables were compared with the χ² test or Fisher’s exact test, with results presented as cases and percentages [n (%)]. Five randomized imputations were performed on the missing data using the mice package. For continuous variables, the mean value was taken based on the results of randomized imputations, while for categorical variables the mode was adopted to fill the missing values and to generate the final dataset.

Absence or presence of in-hospital mortality was the outcome and sentiment scores were the major research factor. Comparison among groups was conducted to identify the potential confounding factors, which were then included into the multivariate model through stepwise regression. Additionally, the association between sentiment scores and in-hospital mortality was also analyzed in multivariate logistic models, especially when the remaining variables screened by the stepwise regression were adjusted.

All patients were randomly assigned into the training group (n=410) and the testing group (n=221) with a proportion of 6.5:3.5. A predictive model for in-hospital mortality was developed in the training group using logistic regression analysis and then validated in the testing group. Various indicators, including the area under the curve (AUC), sensitivity and specificity, were used to assess the performance of this predictive model. Delong test was utilized to compare the predictive performance of this model, this model without sentiment scores, SOFA score, and SAPS-II score. The AUC value was over 0.8, indicating the performance of the model was good. Moreover, the clinical value of the predictive model was also examined using decision curves. The statistical power was calculated by PASS software (NCSS, Kaysville, Utah, USA; version 15), and the sample sizes of the training and testing samples were adequate for assessing the predictive validity. A two-sided P value less than 0.05 was considered statistically significant. All the data were managed by SAS software (SAS Institute Inc., Cary, NC, USA; version 9.4) and R package (version 3.6.3).

Results

Baseline information of AP patients

A total of 904 AP patients were identified in the MIMIC-III database. After excluding 2 cases aged <18 years and 271 cases that did not have clinical notes, 631 patients were finally included into the study for analysis. Among them, 543 cases (86.1%) survived, while 88 (13.9%) died during hospitalization. The baseline information of the surviving and dead patients is compared in Table 1.

Results showed that there were significant differences in multiple variables, such as age (P<0.001), ICU LOS (P<0.001), respiratory rate (P=0.009), temperature (P=0.002), systolic blood pressure (P=0.036), diastolic blood pressure (P=0.011), mean arterial pressure (P=0.017), red blood cells (P=0.003), phosphate (P<0.001), INR (P=0.007), MCV (P=0.036), creatinine (P=0.046), BUN (P<0.001), lymphocytes (P=0.004), albumin (P<0.001), TBIL (P=0.001), hematocrit (P=0.025), hemoglobin (P=0.004), MCHC (P<0.001), ALP (P=0.033), RDW (P<0.001), history of lung cancer (P=0.009), atrial fibrillation (P=0.013), liver cirrhosis (P=0.014), respiratory failure (P<0.001), hyperlipidemia (P<0.001) and renal failure (P<0.001), SAPS-II (P<0.001), SOFA scores (P<0.001), subjective minimum (P=0.017), polarity mean (P<0.001), and polarity minimum (P<0.001).

Association between sentiment scores and in-hospital morality

As shown in Table 2, the correlation between sentiment scores and in-hospital mortality was analyzed in each of the three models. In the original model (model 1), subjective minimum [odds ratio (OR): 0.819; 95% confidence interval (CI): 0.694–0.966; P=0.018], polarity mean (OR: 0.353; 95% CI: 0.210–0.577; P<0.001), and polarity minimum (OR: 0.756; 95% CI: 0.651–0.888; P<0.001) were all shown to be protective factors for in-hospital mortality in AP patients. After the confounding factors including age, gender, ethnicity, and ICU LOS were corrected, polarity mean (OR: 0.391; 95% CI: 0.226–0.656; P<0.001; model 2) was found to be associated with a reduced risk of in-hospital mortality. After adjusting for confounding factors in model 2 and the remaining covariates screened by stepwise regression, such as respiratory rate, TBIL, hemoglobin, lung cancer, respiratory failure, hyperlipidemia and renal failure, polarity mean (OR: 0.448; 95% CI: 0.233–0.833; P=0.014; model 3) remained an independent protective factor for in-hospital mortality in AP patients, which was consistent with the results before data imputation (Table 3).

Establishment of a predictive model for in-hospital mortality

According to the proportion of 6.5:3.5, the patients were divided into the training group (n=410) and the testing group (n=221). In comparison to the baseline characteristics, no significant differences were identified between the training group and the testing group (all P>0.05; Table 4), which suggested balanced and comparable data between these two groups.

Using all variables in model 3 as the predictors, a predictive model for in-hospital mortality was established in the training group via logistic regression analysis, namely ln(p/1-p) = −4.61 − 1.06005 × polarity mean + 0.04 × age + 0.54 × gender (male) −1.116 × Asian + 0.71 × Black − 0.33 × Hispanic-14.88 × others + 0.02 × ICU LOS + 0.05 × respiratory rate + 0.07 × TBIL-0.11 × hemoglobin + 3.92 × lung cancer + 0.55 × respiratory failure − 1.56 × hyperlipidemia + 0.69 × renal failure. When this model was validated in the testing group, it showed a sensitivity of 0.656, a specificity of 0.815, and an AUC of 0.812 (Table 5).

Performance and clinical value of the predictive model

The assessment indicators of sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), AUC, and accuracy were used to compare the performance of this predictive model, this predictive model without sentiment scores, SOFA score and SAPS-II score in the prediction of in-hospital mortality. As shown in Table 5, the AUC of this model was up to 0.840 in the training group and 0.812 in the testing group, which was significantly higher than those of this predictive model without sentiment scores (0.759 and 0.793; P<0.05), SOFA (0.661 and 0.584; P<0.05) and SAPS-II (0.725 and 0.792; P<0.05). Additionally, the specificity and accuracy of the predictive model were both superior to SOFA (training group, specificity: 0.760 vs. 0.540, accuracy: 0.763 vs. 0.561, P<0.05; testing group; specificity: 0.760 vs. 0.540, accuracy: 0.763 vs. 0.561, P<0.05). The receiver operating characteristic curves of the different models in the prediction of in-hospital mortality is shown in Figure 1.

Figure 1 The receiver operating characteristic curves of different models in the prediction of in-hospital mortality. (A) Training group; (B) testing group. AUC, area under the curve; SOFA, sequential organ failure assessment; SAPS-II, Simplified Acute Physiology Score-II.

Decision-making curves were used to compare the clinical value of different models. Results showed that, under the same level and types of risk, the clinical benefits of the predictive model were the highest compared with SOFA and SAPS-II (Figure 2).

Figure 2 Comparison of the clinical value among different models. SOFA, sequential organ failure assessment; SAPS-II, Simplified Acute Physiology Score-II.

A predictive nomogram for in-hospital mortality

A nomogram was constructed based on the predictors included in the predictive model (Figure 3). The Hosmer-Lemeshow test of goodness of fit was performed on the predictive model. The P value of goodness of fit was 0.807, larger than 0.05, which suggested a good fit ability of the predictive model.

Figure 3 A nomogram to predict the risk of in-hospital mortality in acute pancreatitis. *, P<0.05; **, P<0.01; ***, P<0.001.

For example, the 15^th patient from the dataset was a Hispanic female aged 24 years, with ICU LOS of 1 day and respiratory rate of 24 times/min. The levels of hemoglobin and TBIL were 9.8 g/dL and 0.6 mg/dL, respectively. She had renal failure, hyperlipidemia, and respiratory failure, but no lung cancer. The final predictive scores were 945, with the probability of in-hospital mortality of 0.0185 (Figure 4). This patient survived, which suggested that the nomogram in predicting in-hospital mortality was accurate.

Figure 4 A schematic nomogram for predicting the risk of in-hospital mortality in a specific patient with acute pancreatitis. *, P<0.05; **, P<0.01; ***, P<0.001.

Discussion

In this study, a total of 631 AP patients were eligible for analysis, with the in-hospital mortality rate of 13.9%. Analysis of the association between sentiment scores and in-hospital mortality indicated by the sentiment polarity mean was correlated with a lowered risk of in-hospital mortality. Through multivariate logistic regression analysis, 12 independent variables were screened out to establish a predictive model for in-hospital mortality, which represented better performance and clinical value than other models. Additionally, a nomogram constructed based on the predictors further demonstrated the accuracy of the predictive model for in-hospital mortality of AP patients.

Severity of disease scoring systems, such as SOFA and SAPS II, are usually used to predict mortality in the ICU. These scoring systems are computed through the coded data of laboratory test results, demographics, and vital signs accessed from the electronic health record (EHR) of patients, but they have poor generalizability across diseases and countries (15,21,22). A total of 80% of all EHRs contain unstructured data, such as clinician-written notes with crucial information on the physical condition and trajectory of patients (23,24). Although the unstructured form of clinical notes enables clinicians to document rich, accurate, and domain-specific information, they are difficult to quantify and are easy to neglect in the structured fields of the EHR (15,25).

Previous studies indicated that the sentiment in clinical notes can be measured using natural language processing tools and is associated with the likelihood of mortality and readmission to ICU (17). Quantitative assessment of the sentiment in clinical notes related to the 30-day mortality of patients in the ICU, and there was a positive association between the sentiment polarity mean and patients’ survival (19). In this study, Python’s TextBlob library was applied to measure the sentiment in clinical notes via natural language processing and text analysis, and sentiment polarity mean was found to be associated with a reduced risk of in-hospital mortality in AP, similar to the results of Waudby-Smith et al. (19). Based on this, a predictive model for in-hospital mortality in AP was developed via multivariate logistic regression analysis. In the prediction of in-hospital mortality, this predictive model showed the largest AUC and the highest clinical benefits compared to the model without sentiment scores, SOFA and SAPS-II. SOFA score is mainly used to assess organ failure, which enables it to predict clinical outcomes in critically ill patients (26). However, the AUC of SOFA in this study was less than 0.7, which indicated a poor predictive performance for in-hospital mortality. This might be partially explained by the fact that the organ dysfunction was often easy to detect and treat in the early phase of AP, but deterioration of organ dysfunction was correlated with mortality and poor clinical outcomes (27). Based on a large international sample of patients, SAPS-II was developed to estimate the mortality risk without having to specify the primary diagnosis (28). As the most common tool in ICU, SAPS-II consisted of 12 immediate variables that should be used the first 24 hours of admission to the ICU; age and comorbidities obtained prior to admission were also taken into consideration (29). However, it is probable to misdiagnose the severity in the early stage of AP, thereby restricting the application of SAPS-II in the early prognostic assessment of AP (30,31).

To the best of our knowledge, this was the first study to develop a predictive model with well performance and clinical value for in-hospital mortality in AP using the sentiment in clinical notes, which highlighted the potential of unstructured data to improve mortality prediction. Moreover, the visualized nomogram performed well, which further demonstrated the accuracy of this predictive model to assess the prognosis of AP patients. Nevertheless, several limitations mean our findings should be interpreted with caution. First, the data used in our study were obtained from MIMIC-III database where most patients were Americans, therefore this predictive model may have limitations when applied to other ethnicities. Second, some important factors related to poor prognosis of AP patients were missing in MIMIC-III, such as interlukin-6 (32). Third, no external validation was used to further validate the performance of the model. Fourth, despite having a relatively large sample size it was a database-based retrospective study. In the future, more prospective studies with larger samples should be performed to validate our results and to enhance the clinical application of this predictive model for in-hospital mortality in AP.

Conclusions

Our findings demonstrated that sentiment polarity mean was a significant protective factor for in-hospital mortality in AP. Based on this, a predictive model was developed. This model showed well performance and clinical value in the prediction of in-hospital mortality in AP patients.

Acknowledgments

Funding: None.

Footnote

Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at https://atm.amegroups.com/article/view/10.21037/atm-22-1613/rc

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-1613/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013).

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Shi N, Sun GD, Ji YY, et al. Effects of acute kidney injury on acute pancreatitis patients' survival rate in intensive care unit: A retrospective study. World J Gastroenterol 2021;27:6453-64. [Crossref] [PubMed]
2. Lee PJ, Papachristou GI. New insights into acute pancreatitis. Nat Rev Gastroenterol Hepatol 2019;16:479-96. [Crossref] [PubMed]
3. Peery AF, Crockett SD, Barritt AS, et al. Burden of Gastrointestinal, Liver, and Pancreatic Diseases in the United States. Gastroenterology 2015;149:1731-1741.e3. [Crossref] [PubMed]
4. Wadhwa V, Patwardhan S, Garg SK, et al. Health Care Utilization and Costs Associated With Acute Pancreatitis. Pancreas 2017;46:410-5. [Crossref] [PubMed]
5. Banks PA, Bollen TL, Dervenis C, et al. Classification of acute pancreatitis--2012: revision of the Atlanta classification and definitions by international consensus. Gut 2013;62:102-11. [Crossref] [PubMed]
6. Zerem E. Treatment of severe acute pancreatitis and its complications. World J Gastroenterol 2014;20:13879-92. [Crossref] [PubMed]
7. Yasuda H, Horibe M, Sanui M, et al. Etiology and mortality in severe acute pancreatitis: A multicenter study in Japan. Pancreatology 2020;20:307-17. [Crossref] [PubMed]
8. Machicado JD, Gougol A, Stello K, et al. Acute Pancreatitis Has a Long-term Deleterious Effect on Physical Health Related Quality of Life. Clin Gastroenterol Hepatol 2017;15:1435-1443.e2. [Crossref] [PubMed]
9. Zhou H, Mei X, He X, et al. Severity stratification and prognostic prediction of patients with acute pancreatitis at early phase: A retrospective study. Medicine (Baltimore) 2019;98:e15275. [Crossref] [PubMed]
10. Vasudevan S, Goswami P, Sonika U, et al. Comparison of Various Scoring Systems and Biochemical Markers in Predicting the Outcome in Acute Pancreatitis. Pancreas 2018;47:65-71. [Crossref] [PubMed]
11. Chen Y, Tang S, Wang Y. Prognostic Value of Glucose-to-Lymphocyte Ratio in Critically Ill Patients with Acute Pancreatitis. Int J Gen Med 2021;14:5449-60. [Crossref] [PubMed]
12. Han T, Cheng T, Liao Y, et al. Development and Validation of a Novel Prognostic Score Based on Thrombotic and Inflammatory Biomarkers for Predicting 28-Day Adverse Outcomes in Patients with Acute Pancreatitis. J Inflamm Res 2022;15:395-408. [Crossref] [PubMed]
13. Ding N, Guo C, Li C, et al. An Artificial Neural Networks Model for Early Predicting In-Hospital Mortality in Acute Pancreatitis in MIMIC-III. Biomed Res Int 2021;2021:6638919. [Crossref] [PubMed]
14. Zou Y, Wang J, Lei Z, et al. Sentiment analysis for necessary preview of 30-day mortality in sepsis patients and the control strategies. J Healthc Eng 2021;2021:1713363. [Crossref] [PubMed]
15. Hashir M, Sawhney R. Towards unstructured mortality prediction with free-text clinical notes. J Biomed Inform 2020;108:103489. [Crossref] [PubMed]
16. Gao Q, Wang D, Sun P, et al. Sentiment Analysis Based on the Nursing Notes on In-Hospital 28-Day Mortality of Sepsis Patients Utilizing the MIMIC-III Database. Comput Math Methods Med 2021;2021:3440778. [Crossref] [PubMed]
17. McCoy TH, Castro VM, Cagan A, et al. Sentiment Measured in Hospital Discharge Notes Is Associated with Readmission and Mortality Risk: An Electronic Health Record Study. PLoS One 2015;10:e0136341. [Crossref] [PubMed]
18. Johnson AE, Pollard TJ, Shen L, et al. MIMIC-III, a freely accessible critical care database. Sci Data 2016;3:160035. [Crossref] [PubMed]
19. Waudby-Smith IER, Tran N, Dubin JA, et al. Sentiment in nursing notes as an indicator of out-of-hospital mortality in intensive care patients. PLoS One 2018;13:e0198687. [Crossref] [PubMed]
20. Saleh SN, Lehmann CU, McDonald SA, et al. Understanding public perception of coronavirus disease 2019 (COVID-19) social distancing on Twitter. Infect Control Hosp Epidemiol 2021;42:131-8. [Crossref] [PubMed]
21. Haniffa R, Isaam I, De Silva AP, et al. Performance of critical care prognostic scoring systems in low and middle-income countries: a systematic review. Crit Care 2018;22:18. [Crossref] [PubMed]
22. Cheng JY. Mortality prediction in status epilepticus with the APACHE II score. J Intensive Care Soc 2017;18:310-7. [Crossref] [PubMed]
23. Murdoch TB, Detsky AS. The inevitable application of big data to health care. JAMA 2013;309:1351-2. [Crossref] [PubMed]
24. Boag Willie, Doss Dustin, Naumann Tristan, et al. What’s in a note? unpacking predictive value in clinical note representations. AMIA Jt Summits Transl Sci Proc 2017;26-34. [PubMed]
25. Aylor M, Campbell EM, Winter C, et al. Resident Notes in an Electronic Health Record. Clin Pediatr (Phila) 2017;56:257-62. [Crossref] [PubMed]
26. Lambden S, Laterre PF, Levy MM, et al. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care 2019;23:374. [Crossref] [PubMed]
27. Yadav J, Yadav SK, Kumar S, et al. Predicting morbidity and mortality in acute pancreatitis in an Indian population: a comparative study of the BISAP score, Ranson's score and CT severity index. Gastroenterol Rep (Oxf) 2016;4:216-20. [Crossref] [PubMed]
28. Haq A, Patil S, Parcells AL, et al. The Simplified Acute Physiology Score III Is Superior to the Simplified Acute Physiology Score II and Acute Physiology and Chronic Health Evaluation II in Predicting Surgical and ICU Mortality in the "Oldest Old". Curr Gerontol Geriatr Res 2014;2014:934852. [Crossref] [PubMed]
29. Ferreira Ade F, Bartelega JA, Urbano HC, et al. Acute pancreatitis gravity predictive factors: which and when to use them? Arq Bras Cir Dig 2015;28:207-11. [Crossref] [PubMed]
30. Domínguez-Muñoz JE, Carballo F, García MJ, et al. Evaluation of the clinical usefulness of APACHE II and SAPS systems in the initial prognostic classification of acute pancreatitis: a multicenter study. Pancreas 1993;8:682-6. [Crossref] [PubMed]
31. Capuzzo M, Moreno RP, Le Gall JR. Outcome prediction in critical care: the Simplified Acute Physiology Score models. Curr Opin Crit Care 2008;14:485-90. [Crossref] [PubMed]
32. Arutla M, Raghunath M, Deepika G, et al. Efficacy of enteral glutamine supplementation in patients with severe and predicted severe acute pancreatitis- A randomized controlled trial. Indian J Gastroenterol 2019;38:338-47. [Crossref] [PubMed]

(English Language Editor: C. Mullens)