Development and validation of the prediction models for preeclampsia: a retrospective, single-center, case-control study

Introduction

Preeclampsia (PE) is a severe complication that occurs in approximately 4–5% of all pregnancies worldwide (1-5). It is characterized by new-onset hypertension and proteinuria after 20 weeks of gestation in healthy pregnant women (4-7). Due to the ability to progress to multiorgan dysfunction, including acute kidney disease, eclampsia, myocardial ischemia, and other life-threatening complications, PE is related to substantial maternal morbidity and mortality (4-6,8). It is also associated with significant fetal morbidity and mortality, including intrauterine growth restriction, placental abruption, oligohydramnios, stillbirth, and fetal death (4,5). The exact etiology of PE remains unclear (4), but hypotheses include abnormal placental implantation (4), angiogenic pathways (4,9,10), cardiovascular maladaptation and vasoconstriction (4), genetic predisposition (4), immunology (4), oxidative stress (4,5,11), autocrine/paracrine factors (12,13), and capillary rarefaction (14). Clinical practice guidelines strongly recommend low-dose aspirin at 12–16 weeks’ gestation for pregnant women at high risk of PE (15,16). Blood pressure (BP) must be controlled in women with PE (6,17). The management and delivery of patients with PE require special considerations (6).

An accurate diagnosis of PE using clinically accessible risk factors in the early second trimester is crucial to the prevention of PE progression. At present, many studies have assessed the predictive ability of various factors for predicting PE, including clinical features, biomarkers, and ultrasound markers (18-24). Nevertheless, no consensus has been reached on the best strategy for predicting the likelihood of PE (6,25). Kim et al. developed a prediction model of PE on the basis of maternal characteristics and serum markers among twin pregnancy women, with the area under the curve (AUC) of 0.73 by a logistic regression (LR) model (18). Han et al. mentioned a model based on the peripheral blood test value to predicting PE among 568 Chinese women, with the C-index of 0.73 via a LR model (26). The classification tree (CT) model and random forest (RF) algorithm have been widely applied in clinical research (27,28). However, there are limited reports on the performance of these models in predicting the risk of PE.

In this current study, we investigated the predictors associated with the occurrence of PE, and developed the prediction models to predict the risk of PE, then compared the performance of the models, which may help clinicians to identify the high-risk women to improve the perinatal outcomes. We present the following article in accordance with the STARD reporting checklist (available at https://atm.amegroups.com/article/view/10.21037/atm-22-4192/rc).

Methods

Study design and patients

This monocentric, retrospective, case-control analysis included pregnant women who received antenatal care and delivered at the Department of Obstetrics in the Second Hospital of Tianjin Medical University between October 2018 and July 2020. The inclusion criterion was pregnant women who gave birth at the Second Hospital of Tianjin Medical University. The exclusion criteria were as follows: (I) the pregnancy was terminated before 24 weeks of pregnancy and (II) patients who did not give birth at the study hospital. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Institutional Research Ethics Committee of the Second Hospital of Tianjin Medical University (#KY2021K060). The requirement for informed consent was waived by the committee due to the retrospective nature of the study.

Data collection

Maternal information was accessed from the hospital’s electronic medical records. Maternal age, primiparity, multiple births, history of smoking and drinking, history of PE, history of abnormal gestation, pre-pregnancy body mass index (pBMI), BP ≥130/80 mmHg in early pregnancy, assisted reproduction, comorbidities [chronic hypertension, diabetes mellitus (DM), gestational diabetes mellitus (GDM) and chronic kidney disease], duration of hypertension, assisted reproduction, and family history of hypertension and DM were collected from all women.

Development and validation of the prediction models

All women were randomized 3:1 to the training and testing sets. Three models of logistic regression (LR), CT, and RF were developed to predict the risk of PE using the training set. Univariable and multivariable LR analyses were performed to screen out the statistically significant independent predictors. Then, the LR model for predicting the PE was performed using the predictors. Both of CT and RF are emerging and flexible machine learning methods that can be used to predict disease risk and patient susceptibility. The CT is a kind of supervised learning. Each patient has a set of characteristics and a predetermined category. The classifier is obtained by learning, after which the newly emerged objects can be correctly classified. The predictive factors of CT model included all the clinical variables mentioned above. The RF establishes a forest in a random way; its basic unit is the CT. In this study, the number of CTs in the RF was 10,000 (tree =10,000). The RF sorts the included predictive factors in order of importance. Receiver operator characteristic (ROC) curves of the three models were drew. The predictive performance of the models was accessed by the AUC values, 95% confidence intervals (95% CIs) sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). The comparison of the three models were conducted by the DeLong test.

Outcomes

The occurrence of PE during the perinatal period was considered the primary endpoint. Diagnosis of PE was made according to the 2013 American College of Obstetricians and Gynecologists (ACOG) criteria (29).

Statistical analysis

The Kolmogorov-Smirnov test was run for the determination of the normal distribution of data. Continuous variables with a normal or non-normal distribution were expressed as means ± standard deviations or medians (ranges). Continuous variables with a normal distribution (or a non-normal distribution) were analyzed using the independent samples t-test (or the Mann-Whitney U test). Categorical data were expressed as n (%). The chi-square test was used to compare the categorical variables. The LR was performed with using SPSS 20.0 (IBM Corp., Armonk, NY, USA). The CT and RF were performed with the R rpart package and the RF package, respectively (http://cran.r-project.org/bin/windows/base/). The DeLong test was used to compare the performance of three prediction models via MedCalc 19.0.4 (MedCalc Software Ltd., Ostend, Belgium). The point with the highest Youden index on the ROC curve of the training set (i.e., specificity + sensitivity/1) was determined as the optimal critical value. Two-sided P<0.05 was considered statistical difference.

Results

Participant characteristics

The characteristics of all participants are shown in Table 1. There were no differences between the training and test sets (all P>0.05). In the training set, compared with the non-PE group, the patients with PE had a longer duration of hypertension (0.6±2.0 vs. 0.1±0.6 years, P<0.001), higher frequencies of multiple gestations (9.6% vs. 5.1%, P=0.026), GDM (46.3% vs. 29.9%, P<0.001), chronic hypertension (23.9% vs. 2.4%, P<0.001), DM (4.8% vs. 1.8%, P=0.028), kidney diseases (2.7% vs. 0.4%, P=0.008), history of PE (8.5% vs. 0.4%, P<0.001), BP ≥130/80 mmHg (64.4% vs. 13.6%, P<0.001), family history of hypertension (75.5% vs. 19.7%, P<0.001), and higher pBMI (27.4±5.4 vs. 22.9±3.5 kg/m², P<0.001). In the test set, compared with the control group, the patients with PE were older (31.5±5.7 vs. 30.5±4.1 years, P=0.023), had a longer duration of hypertension (0.8±2.5 vs. 0.1±1.1 years, P<0.001), higher frequencies of chronic hypertension (32.7% vs. 1.1%, P<0.001), history of PE (8.2% vs. 0.5%, P=0.006), BP ≥130/80 mmHg (77.6% vs. 11.2%, P<0.001), family history of hypertension (83.3% vs. 13.4%, P<0.001), and higher pBMI (26.6±3.8 vs. 23.2±3.3 kg/m², P<0.001).

Development of the prediction models

Table 2 shows the correlation strength between risk factors and PE obtained by the multivariable logistic regression analysis. Kidney disease, history of PE, BP ≥130/80 mmHg in early pregnancy, chronic hypertension, family history of hypertension, and pBMI significantly contributed.

Figure 1 shows the CT decision tree, including a family history of hypertension, BP ≥130/80 mmHg in early pregnancy, pBMI, duration of hypertension, and family history of DM. The most important factor was a family history of hypertension, followed by BP ≥130/80 mmHg in early pregnancy.

Figure 1 The CT model for predicting the PE. CT, classification tree; PE, preeclampsia; BP, blood pressure in the early pregnancy; pBMI, pre-pregnancy body mass index; DM, diabetes mellitus.

The importance of variables for predicting PE was analyzed using RF. Among these variables, the six essential factors in the predictive model were family history of hypertension, pBMI, BP ≥130/80 mmHg in early pregnancy, age, chronic hypertension, and duration of hypertension (Figure 2).

Figure 2 The variable importance of the RF model for predicting the PE. x-axis is mean decrease Gini index. RF, random forest; PE, preeclampsia; pBMI, pre-pregnancy body mass index; BP, blood pressure in the early pregnancy.

Validation of the prediction models

Figure 3 shows the calibration plots with the respective AUCs of the LR, CT, and RF models. The AUC was highest in the RF model (AUC =0.871). The AUC values for the LR and CT were 0.778 and 0.850, respectively (all P<0.05 for all pair-wise comparisons). When the prediction performances of each model were compared, the RF model was shown to perform the best. The sensitivity, specificity, PPV, and NPV of the RF model for predicting the risk of PE were 79.6%, 94.7%, 79.6%, and 94.7%, respectively (Table 3).

Figure 3 AUC of different models based on the testing set. Logistic regression model (AUC =0.778; 95% CI: 0.719 to 0.829); classification tree model (AUC =0.850: 95% CI: 0.798 to 0.893); random forests model (AUC =0.871; 95% CI: 0.822-0.911). AUC, area under the ROC curve; ROC, receiver operating characteristic; LR, logistic regression; CT, classification tree; RF, random forest.

Discussion

This study aimed to develop a new data mining model for predicting PE in early and mid-pregnancy. The results suggest that the RF model could be a practical screening approach for predicting PE in the early second trimester of pregnancy. Large-scale studies are needed to validate its clinical applicability.

The pathogenesis of PE is complex, and its etiology is somewhat unclear (1,30,31). Numerous studies have attempted to assess the predictive performance of various factors for PE, including clinical features, biomarkers, and ultrasound indicators (18-24). Identification of early risk factors is the best way to prevent PE. However, risk factors can sometimes only be seen in mid and late gestation, hence the importance of identifying early markers.

Maternal data in the early second trimester are crucial for accurate PE prediction, but the acquisition of many risk variables is not practical during the antenatal examination (early second trimester), and biomarkers that are not examined routinely cannot be used for a screening purpose. Therefore, there is a need for a concise, evidence-based, and effective tool that clinicians can use to screen women at high risk of PE in the early second trimester. This study’s prediction models used readily available clinical data during early-to-mid pregnancy to effectively predict PE’s onset. The RF algorithm had the best predictive efficiency with high accuracy and a low false-positive rate. Therefore, the RF model should be explored as a practical screening method for PE in future studies.

Previous studies have found that maternal age ≥35 years, family history of hypertension or DM, history of PE, chronic hypertension, duration of hypertension, GDM, and pre-pregnancy BMI ≥30 kg/m² are all associated with an increased incidence of PE (18-24,32,33). In this study, kidney disease and a family history of PE was shown to be the most important factor in predicting PE, followed by a family history of hypertension, pBMI ≥27.7 kg/m², and BP ≥130/80 mmHg in the first trimester. Bartsch et al. (16) reported that PE history is the most important risk factor for PE. These discrepancies might be explained by the long interval between two pregnancies, resulting in patients’ unclear descriptions or memory of previous medical history when collecting clinical data. In addition, the risk factors might differ among populations.

Numerous studies have demonstrated that obesity (BMI ≥30 kg/m²) is closely associated with a higher PE (16,32,34). In this study, pBMI ≥27.7 kg/m² was selected, which may be related to the different epidemiological characteristics of obesity and disease risk data of Chinese women. The diagnostic criterion for obesity in China is BMI ≥28 kg/m² (35). Jhee et al. (32) reported that systolic and diastolic BP in patients with PE were higher compared to those who were not in the early and middle stages of pregnancy (116.7±12.3/71.6±9.2 and 111.73±8.7/67.8±6.5 mmHg, respectively). Moreover, Ye et al. (36) found that C hypertension was an independent risk factor for PE and the incidence of PE was 3.6 times higher in patients with systolic BP >130 mmHg in the first prenatal examination compared with those with systolic BP <110 mmHg. In the present study, BP ≥130/80 mmHg in the first trimester was significant factor for predicting PE, which is consistent with the above reports.

This study had some limitations. First, compared to the control group, the number of PE cases was relatively smaller. More clinical cases are needed to improve the accuracy rate of the predictive models. Second, the selected variables were mainly limited to maternal medical history characteristics. To improve the models’ predictive performance, some biological and ultrasonic indicators that can be easily obtained in the early and middle stages of pregnancy could be used. Finally, based on the predictive weight of PE’s high-risk factors, a complete scoring system and prospective studies are needed to verify the prediction models’ accuracy further.

Conclusions

In conclusion, the RF model could be a practical screening approach for predicting PE in the early second trimester of pregnancy. Pregnant women’s family history of hypertension, pBMI, BP ≥130/80 mmHg in early pregnancy, age, chronic hypertension, and duration of hypertension can be used as indicators to predict the risk of PE. Large-scale studies are needed to validate its clinical applicability.

Acknowledgments

Funding: This work was supported by the Science and Technology Foundation of Health and Family Planning Commission of Tianjin Binhai New Area (grant number 2018BWKY013 to XHC). The funding organization(s) played no role in the study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the report for publication.

Footnote

Reporting Checklist: The authors have completed the STARD reporting checklist. Available at https://atm.amegroups.com/article/view/10.21037/atm-22-4192/rc

Data Sharing Statement: Available at https://atm.amegroups.com/article/view/10.21037/atm-22-4192/dss

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://atm.amegroups.com/article/view/10.21037/atm-22-4192/coif). XC reports that this work was supported by the Science and Technology Foundation of Health and Family Planning Commission of Tianjin Binhai New Area (grant number 2018BWKY013). The other authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). This study was approved by the Institutional Research Ethics Committee of the Second Hospital of Tianjin Medical University (#KY2021K060). The requirement for informed consent was waived by the committee due to the retrospective nature of the study.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Park YS, Kim Y, Kim HY, et al. Serum sFlt-1, cystatin C and cathepsin B are potential severity markers in preeclampsia: a pilot study. Arch Gynecol Obstet 2020;301:955-62. [Crossref] [PubMed]
2. Phipps EA, Thadhani R, Benzing T, et al. Pre-eclampsia: pathogenesis, novel diagnostics and therapies. Nat Rev Nephrol 2019;15:275-89. [Crossref] [PubMed]
3. Yang Y, Le Ray I, Zhu J, et al. Preeclampsia prevalence, risk factors, and pregnancy outcomes in Sweden and China. JAMA Netw Open 2021;4:e218401. [Crossref] [PubMed]
4. Metoki H, Iwama N, Hamada H, et al. Hypertensive disorders of pregnancy: definition, management, and out-of-office blood pressure measurement. Hypertens Res 2022;45:1298-309. [Crossref] [PubMed]
5. Li F, Wang T, Chen L, et al. Adverse pregnancy outcomes among mothers with hypertensive disorders in pregnancy: A meta-analysis of cohort studies. Pregnancy Hypertens 2021;24:107-17. [Crossref] [PubMed]
6. ACOG Practice Bulletin No. 202: Gestational Hypertension and Preeclampsia. Obstet Gynecol 2019;133:1.
7. American College of Obstetricians and Gynecologists' Committee on Practice Bulletins—Obstetrics. ACOG Practice Bulletin No.203: Chronic Hypertension in Pregnancy. Obstet Gynecol 2019;133:e26-50. [Crossref] [PubMed]
8. Covella B, Vinturache AE, Cabiddu G, et al. A systematic review and meta-analysis indicates long-term risk of chronic and end-stage kidney disease after preeclampsia. Kidney Int 2019;96:711-27. [Crossref] [PubMed]
9. Rana S, Burke SD, Karumanchi SA. Imbalances in circulating angiogenic factors in the pathophysiology of preeclampsia and related disorders. Am J Obstet Gynecol 2022;226:S1019-34. [Crossref] [PubMed]
10. Verlohren S, Dröge LA. The diagnostic value of angiogenic and antiangiogenic factors in differential diagnosis of preeclampsia. Am J Obstet Gynecol 2022;226:S1048-58. [Crossref] [PubMed]
11. Stanhewicz AE, Dillon GA, Serviente C, et al. Acute systemic inhibition of inflammation augments endothelium-dependent dilation in women with a history of preeclamptic pregnancy. Pregnancy Hypertens 2022;27:81-6. [Crossref] [PubMed]
12. Temur M, Serpim G, Tuzluoğlu S, et al. Comparison of serum human pregnancy-specific beta-1-glycoprotein 1 levels in pregnant women with or without preeclampsia. J Obstet Gynaecol 2020;40:1074-8. [Crossref] [PubMed]
13. Spradley FT. Sympathetic nervous system control of vascular function and blood pressure during pregnancy and preeclampsia. J Hypertens 2019;37:476-87. [Crossref] [PubMed]
14. Kristof T, Merve D, Jerome C, et al. Nailfold video capillaroscopy in pregnant women with and without cardiovascular risk factors. Front Med (Lausanne) 2022;9:904373. [Crossref] [PubMed]
15. US Preventive Services Task Force. Aspirin use to prevent preeclampsia and related morbidity and mortality: US Preventive Services Task Force Recommendation Statement. JAMA 2021;326:1186-91. [Crossref] [PubMed]
16. Bartsch E, Medcalf KE, Park AL, et al. Clinical risk factors for pre-eclampsia determined in early pregnancy: systematic review and meta-analysis of large cohort studies. BMJ 2016;353:i1753. [Crossref] [PubMed]
17. World Health Organization. WHO recommendations. Drug treatment for severe hypertension in pregnancy. Geneva: World Health Organization; 2018.
18. Kim YR, Jung I, Heo SJ, et al. A preeclampsia risk prediction model based on maternal characteristics and serum markers in twin pregnancy. J Matern Fetal Neonatal Med 2021;34:3623-8. [Crossref] [PubMed]
19. Townsend R, Khalil A, Premakumar Y, et al. Prediction of pre-eclampsia: review of reviews. Ultrasound Obstet Gynecol 2019;54:16-27. [Crossref] [PubMed]
20. Li S, Wang Z, Vieira LA, et al. Improving preeclampsia risk prediction by modeling pregnancy trajectories from routinely collected electronic medical record data. NPJ Digit Med 2022;5:68. [Crossref] [PubMed]
21. Salazar MR, Espeche WG, Leiva Sisnieguez CE, et al. Masked hypertension and neonatal outcome in high-risk pregnancies. J Hum Hypertens 2022; Epub ahead of print. [Crossref] [PubMed]
22. Tang Z, Ji Y, Zhou S, et al. Development and validation of multi-stage prediction models for pre-eclampsia: A retrospective cohort study on Chinese women. Front Public Health 2022;10:911975. [Crossref] [PubMed]
23. Corominas AI, Medina Y, Balconi S, et al. Assessing the role of uric acid as a predictor of preeclampsia. Front Physiol 2021;12:785219. [Crossref] [PubMed]
24. Stefańska K, Zieliński M, Zamkowska D, et al. Comparisons of dipstick test, urine protein-to-creatine ratio, and total protein measurement for the diagnosis of Preeclampsia. Int J Environ Res Public Health 2020;17:4195. [Crossref] [PubMed]
25. Brown MA, Magee LA, Kenny LC, et al. The hypertensive disorders of pregnancy: ISSHP classification, diagnosis & management recommendations for international practice. Pregnancy Hypertens 2018;13:291-310. [Crossref] [PubMed]
26. Han Q, Zheng W, Guo XD, et al. A new predicting model of preeclampsia based on peripheral blood test value. Eur Rev Med Pharmacol Sci 2020;24:7222-9. [PubMed]
27. Frizzell JD, Liang L, Schulte PJ, et al. Prediction of 30-Day All-Cause Readmissions in Patients Hospitalized for Heart Failure: Comparison of Machine Learning and Other Statistical Approaches. JAMA Cardiol 2017;2:204-9. [Crossref] [PubMed]
28. Dai P, Chang W, Xin Z, et al. Retrospective study on the influencing factors and prediction of hospitalization expenses for chronic renal failure in China based on random forest and LASSO regression. Front Public Health 2021;9:678276. [Crossref] [PubMed]
29. Hypertension in pregnancy. Report of the American College of Obstetricians and Gynecologists’ Task Force on Hypertension in Pregnancy. Obstet Gynecol 2013;122:1122-31. [PubMed]
30. Phipps E, Prasanna D, Brima W, et al. Preeclampsia: Updates in Pathogenesis, Definitions, and Guidelines. Clin J Am Soc Nephrol 2016;11:1102-13. [Crossref] [PubMed]
31. Iriyama T, Wang W, Parchim NF, et al. Reciprocal upregulation of hypoxia-inducible factor-1α and persistently enhanced placental adenosine signaling contribute to the pathogenesis of preeclampsia. FASEB J 2020;34:4041-54. [Crossref] [PubMed]
32. Jhee JH, Lee S, Park Y, et al. Prediction model development of late-onset preeclampsia using machine learning-based methods. PLoS One 2019;14:e0221202. [Crossref] [PubMed]
33. Al Khalaf SY, O'Reilly ÉJ, McCarthy FP, et al. Pregnancy outcomes in women with chronic kidney disease and chronic hypertension: a National cohort study. Am J Obstet Gynecol 2021;225:298.e1-298.e20. [Crossref] [PubMed]
34. Simko M, Totka A, Vondrova D, et al. Maternal body mass index and gestational weight gain and their association with pregnancy complications and perinatal conditions. Int J Environ Res Public Health 2019;16:1751. [Crossref] [PubMed]
35. Expert Consultation WHO. Appropriate body-mass index for Asian populations and its implications for policy and intervention strategies. Lancet 2004;363:157-63. [Crossref] [PubMed]
36. Ye C, Ruan Y, Zou L, et al. The 2011 survey on hypertensive disorders of pregnancy (HDP) in China: prevalence, risk factors, complications, pregnancy and perinatal outcomes. PLoS One 2014;9:e100180. [Crossref] [PubMed]

(English Language Editor: J. Jones)