A risk score to predict in-hospital mortality in patients with acute coronary syndrome at early medical contact: results from the Improving Care for Cardiovascular Disease in China-Acute Coronary Syndrome (CCC-ACS) Project
Introduction
Ischemic heart disease (IHD) is the leading cause of death globally (1,2). In 2018, the annual mortality ratio among Chinese patients with IHD exceeded 110/100,000, and it is steadily increasing (3). Acute coronary syndrome (ACS) is a severe manifestation of IHD with a prognosis that varies significantly among patients. Therefore, risk stratification is critical for decision-making and management implementation, such as timely invasive strategies for patients at high risk.
Several risk scores for ST-segment elevation myocardial infarction (STEMI), non–ST-segment elevation ACS (NSTE-ACS), and unselected ACS have been developed (4-8), among which some have been recommended by clinical guidelines (9-12). However, the existing risk score systems have some limitations (13). Firstly, most of them were developed prior to or during the early phase of the drug-eluting stent era, and minority of patients underwent percutaneous intervention, thus the discrimination power was relatively poor in those patients. Secondly, acquiring the variables for these risk scores is time consuming, which limits their utility at the point of early medical contact. Further, some risk scores at early medical contact were available, however some ACS patients at high risk were excluded in the registries developing risk score.
The present study aimed to develop and validate a simple and accurate risk score to predict in-hospital death in unselected patients with ACS at early medical contact by using data from the CCC-ACS registry, which represents the real-world practice in the drug-eluting stent era. We present the following article in accordance with the TRIPOD reporting checklist (available at http://dx.doi.org/10.21037/atm-21-31).
Methods
Study protocol
The CCC-ACS project design has been reported previously (14). Briefly, the American Heart Association (AHA) and Chinese Society of Cardiology (CSC) launched the CCC-ACS project in 2014 as a nationwide hospital-based quality improvement registry program to improve the quality of care of patients with ACS. The present study was approval by the Ethics Committee of Beijing Anzhen Hospital, Capital Medical University. As the study used data from a retrospective registry, the requirement for informed consent was waived. All patient information was anonymized and de-identified before analysis. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013).
Study population and data collection
From November 1, 2014 to June 30, 2017, CCC-ACS phases I and II enrolled 63,641 patients with ACS from 150 tertiary hospitals, which represented the highest level of medical care in the 7 geographical regions of China (Northern, Northeast, Eastern, Central, Southern, Southwest, and Northwest China).
Data were collected by trained data abstractors (medical doctors, nurses, medical postgraduates, and clinical research coordinators) at the participating hospitals through a web-based data collection platform (Oracle Clinical Remote Data Capture, Oracle). At each hospital, the first 20–30 ACS patients each month were consecutively enrolled. To ensure that consecutive cases were enrolled, quality audits were performed by third-party clinical research associates. The accuracy and completeness of the clinical data were verified using documents from approximately 5% of enrolled cases, who were randomly selected.
Definitions
Briefly, STEMI and non–ST-segment elevation myocardial infarction (NSTEMI) were defined according to the 2010 CSC STEMI guidelines (15) and the 2012 CSC NSTE-ACS guidelines (16), respectively. Unstable angina (UA) was defined as reported previously (14). Acute heart failure (AHF) and cardiogenic shock (CS) were defined according to the Chinese Guidelines for the Diagnosis and Treatment of Heart Failure 2014 (17), based on the patient’s clinical condition recorded in the medical documentation on hospital admission. The endpoint was in-hospital death. Troponin I (TnI), troponin T (TnT), and creatine kinase MB isoenzyme (CK-MB) elevation was considered when the levels of these markers exceeded the upper level of normal (ULN) of the corresponding local laboratory. Estimated glomerular filtration rate (eGFR) was calculated according to the Modification of Diet in Renal Disease equation.
Statistical analysis
Statistical analyses were performed in SAS (version 9.4, SAS Institute, Cary, North Carolina). Data were presented as the mean ± standard deviation (SD) for normally distributed data, or medians and interquartile ranges (IQR) for non-normally distributed data. Normally and non-normally distributed variables were compared using Student’s t-test and the Mann-Whitney U test, respectively. Categorical data were expressed as numbers (%). Pearson’s χ2 test or Fisher’s exact test were used for categorical data, as appropriate. Using Proc Surveyselect (SAS, SAS Institute, Cary, North Carolina), the simple random sampling method was employed to randomly assign patients to a training dataset or a validation dataset at a ratio of 7:3. The CCC-ACS risk score was constructed by fitting demographic, medical history, clinical, and electrocardiographic variables, which were selected based their clinical significance and the findings of previous studies, as well as on their availability during early medical contact. Variables obtained by laboratory tests were not considered for entry into the model. Potential risk factors were screened through univariate logistic regression analysis with the level of significance set at P<0.05. Independent predictors were identified by performing multivariate logistic regression analysis. Only variables with a P value of <0.05 in the multivariate analysis were entered into the final model. The integer score was generated by multiplying the β coefficient of each selected variable by a constant and rounding the product to the nearest integer. Discrimination and calibration were assessed using the area under the receiver operating characteristic (ROC) curve (AUC) and the Hosmer-Lemeshow (H-L) goodness-of-fit test, respectively. Differences in the discriminatory power between the CCC-ACS score and the Global Registry of Acute Coronary Events (GRACE) score were evaluated using the χ2 test. All P values were 2-tailed, and a P value of <0.05 was considered to represent statistical significance.
Results
There were 63,641 unselected ACS patients analyzed in this study, 44,549 patients initially assigned to the training dataset and 19,092 to the validation dataset. During the modeling process, 775 (1.7%) and 320 (1.7%) patients were excluded from the training and validation cohorts, respectively, due to having missing values for the finally incorporated variables (age, systolic blood pressure, cardiac arrest, and severe clinical conditions). The remaining 43,774 and 18,772 patients were enrolled in the final analyses (Figure 1).
In total, 1,181 in-hospital deaths occurred among the study patients, including 824 (1.9%) in the training dataset and 357 (1.9%) in the validation dataset. As shown in Table 1, except for prior dialysis (0.2% vs. 0.4%, P=0.002), there were no significant differences in demographic, clinical, laboratory, electrocardiographic, or therapeutic characteristics, or in-hospital outcomes between the training and validation cohorts.
Full table
In the training dataset, the in-hospital death group had higher proportions of patients with STEMI, a history of heart failure, hypertension, diabetes mellitus, insulin-treated diabetes mellitus (ITDM), previous dialysis, ST-segment deviation, elevated CK-MB, and 5-fold elevated TNT or TNI. Furthermore, these patients were less likely to smoke or have a history of percutaneous coronary intervention (PCI). Patients in the in-hospital death group in the training dataset were also older, had higher heart rates and serum creatinine levels, and lower systolic blood pressure (SBP), diastolic blood pressure (DBP), and eGFR. Moreover, patients who died in hospital were more likely to present with cardiac arrest, AHF, and CS at admission (Table 2).
Full table
Development and Validation of the CCC-ACS score
The results of univariate and multivariate logistic regression analyses are displayed in Table S1. After univariable and multivariable selection, 7 variables emerged as predictors of mortality, including age, SBP, cardiac arrest, ITDM, history of heart failure, severe clinical conditions at admission (AHF and/or CS), and ST-segment deviation. The scores assigned to each variable based on their estimated β coefficients in the training dataset are shown in Table 3. The AUC for the original model was 0.84, and the χ2 statistic for calibration was 11.48 (P=0.18).
Full table
The scores for each predictor based on their estimated β coefficients are presented in Figure 2. The sum of the score which could theoretically range from 0 to 36, could be used to estimate the risk of in-hospital death for individual patients. In the training dataset, the actual obtained scores ranged from 0 to 31. The CCC-ACS score displayed good discrimination ability (AUC: 0.84) and calibration (χ2=13.43, P=0.10) (Figure 3A). In the validation dataset, the actual obtained scores ranged from 0 to 29, and the CCC-ACS score also displayed good discrimination ability (AUC: 0.85) and calibration (χ2=12.63, P=0.13, Brier score =0.02) (Figure 3B).
Based on the obtained risk scores for in-hospital death, the training dataset was further categorized into the following 3 groups: low risk (score ≤12, n=40,452), moderate risk (score: 13–20, n=2,919), and high risk (score≥21, n=403). The event rate was 0.96%,10.11%, and 34.49%, respectively (Figure 4). The validation dataset was also categorized into 3 groups: low risk (score ≤12, n=17,323), moderate risk (score: 13–20, n=1,269), and high risk (score ≥21, n=180). The event rate was 0.96%,10.01%, and 35.56%, respectively (Figure 4).
Performance in subgroups
The CCC-ACS score also exhibited good discrimination ability after the patients were divided into subgroups according to sex, ACS type, and previous PCI or not (Table S2). After the exclusion of 2,228 patients who had missing values for GRACE variables, the remaining 16,544 patients in the validation dataset were used to compare the performances of the CCC-ACS score and the GRACE score. The 2 scores performed comparably in the prediction of in-hospital death (AUC: CCC-ACS 0.84, 95% CI: 0.81–0.86 vs. GRACE 0.83, 95% CI: 0.81–0.86, P=0.69). The χ2 statistics for the CCC-ACS and GRACE scores were 5.12 (P=0.74) and 8.44 (P=0.39) respectively, showing the good calibration for in-hospital mortality.
Discussion
In the present study, a new in-hospital mortality risk score (CCC-ACS score) was developed and validated. The CCC-ACS risk score comprises 7 variables [age, cardiac arrest, ITDM, history of heart failure, severe clinical conditions at admission (AHF and/or CS), SBP, and ST-segment deviation], and demonstrated good discrimination ability and calibration in predicting the risk of in-hospital death for unselected ACS patients at early medical contact.
Several risk scores have been developed for risk stratification in patients with ACS. Among them, the Thrombolysis in Myocardial Infarction (TIMI) and GRACE scores are recommended by clinical guidelines and are widely applied in clinical practice. Both of these risk scoring systems can provide important information for predicting prognosis and determining the timing of interventions; however, they have some limitations (13). The TIMI risk score was derived from clinical trials and thus has inherent bias due to the exclusion of high-risk patients. The GRACE score was developed from a large-scale unbiased multi-center registry and was validated in external datasets; thus, it has an excellent performance when applied to the general population. Nevertheless, it has been found to lack accuracy for patients undergoing PCI (6), which may because less than 30% of patients in the GRACE (18) and Global Use of Strategies to Open Occluded Coronary Arteries IIB (GUSTO IIB) studies underwent PCI (19,20). Furthermore, in the contemporary era, PCI has been used more widely, and its use has been accompanied by advances in medical treatments, such as P2Y12 antagonist, statin, angiotensin-converting enzyme inhibitors (ACEIs)/angiotensin receptor blockers (ARBs), and β-blockers. In the real-world registry used in the present study, which was compiled in the drug-eluting stent era, 72.3% of ACS patients underwent PCI. Therefore, an updated risk score that is fitting of current clinical practice is needed to supplement the use of previous scoring systems.
The CCC-ACS risk score shares 5 variables (age, cardiac arrest, SBP, severe clinical conditions at admission, and ST-segment deviation) with previous risk scores (4,21), and includes 2 (ITDM and history of heart failure) newly introduced variables. ITDM has been proven as a risk factor for adverse clinical outcomes in patients with NSTE-ACS or those undergoing PCI (22,23). Patients with ITDM may have suffered a longer course of diabetes mellitus and may therefore represent a more severe disease condition (24). History of heart failure, another newly incorporated variable, has also been proved to be associated with in-hospital, 6-month, and 1-year mortality in ACS patients (25-28). A majority of previous studies have focused on AHF in patients with ACS, but a history of heart failure is also important and of independent value. ACS patients with a history of heart failure may have lower cardiac reserve at baseline, and receive evidence-based therapies, such as β-blockers, ACEIs, and PCI, less frequently (25). Although some studies have associated a history of myocardial infarction with adverse outcomes (29,30), it was not found to be an independent predictor after regression in the current analysis. This may be because, at least in part, a history of heart failure is correlated with and more powerful predictor than a history of myocardial infarction. Cardiac markers (TnI, TnT, and CK-MB) and serum creatinine have been demonstrated to be independently associated with adverse outcomes (4,21,31,32), and can improve the discrimination ability of risk scores. However, these markers demand additional time and effort for blood tests to be performed; thus, they are usually not available during early medical contact. In fact, the data of cardiac markers and serum creatinine were lacking for a number of patients in the real-world registry used in the present study.
The main aim of this study was not to replace existing risk scores, but to establish a risk score with variables that are rapidly available at early medical contact. In the emergency unit, where it is busy and risk evaluation needs to be conducted promptly, a risk score based on readily available variables is practically more meaningful. This is also true for ambulance services, community health services, and other facilities with limited medical resources. Although it consists of rapidly obtainable variables, the CCC-ACS risk score displayed similar predictive ability for in-hospital death compared to the GRACE score. In addition, the CCC-ACS risk score exhibited good discrimination ability for those underwent PCI (AUC: 0.84), which is fitting of current clinical practice. Therefore, the CCC-ACS score may serve as a complement to previous risk scores.
There are potential applications of the CCC-ACS risk score. Firstly, stratifying patients at early medical contact without the need for blood tests may facilitate the quick identification of those with the highest risk and, subsequently, their quick and appropriate treatment. Secondly, some identified predictors in this model may provide useful information for updating other ACS risk scores.
Limitations
The present study has several limitations. Firstly, the rate of in-hospital mortality was relatively low among the patients in this study. One explanation was that phase I and phase II of the CCC-ACS project involved only tertiary hospitals, which exhibit a higher standard of patient care than other levels of hospitals. Furthermore, patients who died before or during transfer to the involved hospitals were not included in this study. Secondly, even though the CCC-ACS score was derived from a large-scale dataset, external validation is always required before its general application. Thirdly, the CCC-ACS project is a nationwide hospital-based quality improvement registry program without follow-up data. Therefore, whether the CCC-ACS risk score holds value for long-term prognosis is unknown. This question needs to be solved in further studies with follow-up. Finally, since the data in the CCC-ACS project were obtained from Chinese patients, further investigation is needed to determine whether the risk score performs as well in other populations.
Conclusions
The CCC-ACS CS score, which was developed from a large-scale dataset of unselected ACS patients, can quantify the risk of in-hospital death for patients with ACS at early medical contact and may facilitate clinical decision-making. However, further external validation of this risk score is required.
Acknowledgments
We thank all participating hospitals for their contributions to the CCC-ACS project (Table S3).
Funding: The Improving Care for Cardiovascular Disease in China-Acute Coronary Syndrome (CCC-ACS) project is a collaborative program of the American Heart Association (AHA) and Chinese Society of Cardiology (CSC). The AHA was funded by Pfizer for the quality improvement initiative through an independent grant for learning and change. This work was also supported by Science and Technology Program of Guangzhou (No. 201704020124) and Sailing Foundation (Grant No. LHJJ201611011, LHJJ201612127), Beijing, China.
Footnote
Reporting Checklist: The authors have completed the TRIPOD reporting checklist. Available at http://dx.doi.org/10.21037/atm-21-31
Data Sharing Statement: Available at http://dx.doi.org/10.21037/atm-21-31
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-21-31). Dr. DQY and Dr. PR report grants from Guangzhou Science and Technology Innovation Commission, during the conduct of the study; Dr. GL and Dr. JQY reports grants from Sailing Foundation, during the conduct of the study. The other authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The present study was approval by the Ethics Committee of Beijing Anzhen Hospital, Capital Medical University. All procedures performed in this study involving human participants were in accordance with the Declaration of Helsinki (as revised in 2013). As the study used data from a retrospective registry, the requirement for informed consent was waived.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Roth GA, Forouzanfar MH, Moran AE, et al. Demographic and epidemiologic drivers of global cardiovascular mortality. N Engl J Med 2015;372:1333-41. [Crossref] [PubMed]
- Zhou M, Wang H, Zeng X, et al. Mortality, morbidity, and risk factors in China and its provinces, 1990-2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 2019;394:1145-58. [Crossref] [PubMed]
- Hu S, Gao R, Liu L, et al. Summary of the 2018 Report on Cardiovascular Diseases in China. Chinese Circulation Journal 2019;34:209-20.
- Granger CB, Goldberg RJ, Dabbous O, et al. Predictors of hospital mortality in the global registry of acute coronary events. Arch Intern Med 2003;163:2345-53. [Crossref] [PubMed]
- Antman EM, Cohen M, Bernink PJ, et al. The TIMI risk score for unstable angina/non-ST elevation MI: A method for prognostication and therapeutic decision making. JAMA 2000;284:835-42. [Crossref] [PubMed]
- Palmerini T, Genereux P, Caixeta A, et al. A new score for risk stratification of patients with acute coronary syndromes undergoing percutaneous coronary intervention: the ACUITY-PCI (Acute Catheterization and Urgent Intervention Triage Strategy-Percutaneous Coronary Intervention) risk score. JACC Cardiovasc Interv 2012;5:1108-16. [Crossref] [PubMed]
- Morrow DA, Antman EM, Giugliano RP, et al. A simple risk index for rapid initial triage of patients with ST-elevation myocardial infarction: an InTIME II substudy. LANCET 2001;358:1571-5. [Crossref] [PubMed]
- Huynh T, Kouz S, Yan AT, et al. Canada Acute Coronary Syndrome Risk Score: a new risk score for early prognostication in acute coronary syndromes. Am Heart J 2013;166:58-63. [Crossref] [PubMed]
- Ibanez B, James S, Agewall S, et al. 2017 ESC Guidelines for the management of acute myocardial infarction in patients presenting with ST-segment elevation: The Task Force for the management of acute myocardial infarction in patients presenting with ST-segment elevation of the European Society of Cardiology (ESC). Eur Heart J 2018;39:119-77. [Crossref] [PubMed]
- Roffi M, Patrono C, Collet JP, et al. 2015 ESC Guidelines for the management of acute coronary syndromes in patients presenting without persistent ST-segment elevation: Task Force for the Management of Acute Coronary Syndromes in Patients Presenting without Persistent ST-Segment Elevation of the European Society of Cardiology (ESC). Eur Heart J 2016;37:267-315. [Crossref] [PubMed]
- O'Gara PT, Kushner FG, Ascheim DD, et al. 2013 ACCF/AHA guideline for the management of ST-elevation myocardial infarction: executive summary: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2013;61:485-510. [Crossref] [PubMed]
- Amsterdam EA, Wenger NK, Brindis RG, et al. 2014 AHA/ACC Guideline for the Management of Patients with Non-ST-Elevation Acute Coronary Syndromes: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2014;64:e139-e228. [Crossref] [PubMed]
- Bawamia B, Mehran R, Qiu W, et al. Risk scores in acute coronary syndrome and percutaneous coronary intervention: a review. Am Heart J 2013;165:441-50. [Crossref] [PubMed]
- Hao Y, Liu J, Liu J, et al. Rationale and design of the Improving Care for Cardiovascular Disease in China (CCC) project: A national effort to prompt quality enhancement for acute coronary syndrome. Am Heart J 2016;179:107-15. [Crossref] [PubMed]
- Guideline for diagnosis and treatment of patients with ST-elevation myocardial infarction. Zhonghua Xin Xue Guan Bing Za Zhi 2010;38:675-90. [PubMed]
- Guidelines for the diagnosis and treatment of non–ST-segment elevation acute coronary syndromes. Chin J Cardiol 2012;40:353-67.
- Chinese guidelines for the diagnosis and treatment of heart failure 2014. Zhonghua Xin Xue Guan Bing Za Zhi 2014;42:98-122. [PubMed]
- Steg PG, Goldberg RJ, Gore JM, et al. Baseline characteristics, management practices, and in-hospital outcomes of patients hospitalized with acute coronary syndromes in the Global Registry of Acute Coronary Events (GRACE). Am J Cardiol 2002;90:358-63. [Crossref] [PubMed]
- A comparison of recombinant hirudin with heparin for the treatment of acute coronary syndromes. N Engl J Med 1996;335:775-82. [Crossref] [PubMed]
- Tamis-Holland JE, Palazzo A, Stebbins AL, et al. Benefits of direct angioplasty for women and men with acute myocardial infarction: results of the Global Use of Strategies to Open Occluded Arteries in Acute Coronary Syndromes Angioplasty (GUSTO II-B) Angioplasty Substudy. Am Heart J 2004;147:133-9. [Crossref] [PubMed]
- Chin CT, Chen AY, Wang TY, et al. Risk adjustment for in-hospital mortality of contemporary patients with acute myocardial infarction: the acute coronary treatment and intervention outcomes network (ACTION) registry-get with the guidelines (GWTG) acute myocardial infarction mortality model and risk score. Am Heart J 2011;161:113-22.e2. [Crossref] [PubMed]
- Palmerini T, Dangas G, Mehran R, et al. Predictors and implications of stent thrombosis in non-ST-segment elevation acute coronary syndromes: the ACUITY Trial. Circ Cardiovasc Interv 2011;4:577-84. [Crossref] [PubMed]
- Ritsinger V, Saleh N, Lagerqvist B, et al. High event rate after a first percutaneous coronary intervention in patients with diabetes mellitus: results from the Swedish coronary angiography and angioplasty registry. Circ Cardiovasc Interv 2015;8:e002328. [Crossref] [PubMed]
- Cosmi F, Shen L, Magnoli M, et al. Treatment with insulin is associated with worse outcome in patients with chronic heart failure and diabetes. Eur J Heart Fail 2018;20:888-95. [Crossref] [PubMed]
- Zhang H, Goodman SG, Yan RT, et al. In-hospital management and outcomes of acute coronary syndromes in relation to prior history of heart failure. Eur Heart J Acute Cardiovasc Care 2016;5:214-22. [Crossref] [PubMed]
- Ranasinghe I, Naoum C, Aliprandi-Costa B, et al. Management and outcomes following an acute coronary event in patients with chronic heart failure 1999-2007. Eur J Heart Fail 2012;14:464-72. [Crossref] [PubMed]
- Sanchis J, Nunez J, Bodi V, et al. Influence of comorbid conditions on one-year outcomes in non-ST-segment elevation acute coronary syndrome. Mayo Clin Proc 2011;86:291-6. [Crossref] [PubMed]
- Iakobishvili Z, Feinberg MS, Danicek V, et al. Prior heart failure among patients with acute coronary syndromes is associated with a higher incidence of in-hospital heart failure. Acute Card Care 2006;8:143-7. [Crossref] [PubMed]
- Heggunje PS, Wade MJ, O'Rourke RA, et al. Early invasive versus ischaemia-guided strategies in the management of non-Q wave myocardial infarction patients with and without prior myocardial infarction; results of Veterans Affairs Non-Q Wave Infarction Strategies in Hospital (VANQWISH) trial. Eur Heart J 2000;21:2014-25. [Crossref] [PubMed]
- Sutton AG, Finn P, Hall JA, et al. Predictors of outcome after percutaneous treatment for cardiogenic shock. Heart 2005;91:339-44. [Crossref] [PubMed]
- Bagai A, Schulte PJ, Granger CB, et al. Prognostic implications of creatine kinase-MB measurements in ST-segment elevation myocardial infarction patients treated with primary percutaneous coronary intervention. Am Heart J 2014;168:503-11.e2. [Crossref] [PubMed]
- Galla JM, Mahaffey KW, Sapp SK, et al. Elevated creatine kinase-MB with normal creatine kinase predicts worse outcomes in patients with acute coronary syndromes: results from 4 large clinical trials. Am Heart J 2006;151:16-24. [Crossref] [PubMed]