INTRODUCTION
Suicide among older adults is a critical public health concern worldwide, particularly as the average global population age continues to increase. As of 2021, individuals aged 65 and over account for approximately 10% of the global population and this proportion is expected to increase to around 16% by 2050 [1]. Suicide rates among older adults are the highest, 27.45 per 100,000 people, compared to other age groups worldwide [2]. Compared to other countries in OECD, Korea has the highest suicide rate, 41.7 per 100,000 people, among older adults. This rate is more than double and four times the Korean and global population average, respectively [3]. Various factors contribute to increased suicide risk among older adults, encompassing mental health issues such as depression and cognitive impairments; physical health challenges, including declining physical functions and chronic diseases (CDs); and socioeconomic factors such as economic poverty, social isolation, loneliness, and the loss of close relationships [2].
Suicide is a complex phenomenon that includes ideation, planning, and attempts, and individuals with suicidal ideation (SI) are at a higher risk of dying by suicide [4,5]. Early identification of individuals with SI is crucial for timely intervention to prevent both suicide and suicide attempts effectively. CDs have been reported as strong predictors of both depression and suicide, manifesting through increased symptom burden, pain, disability, and reduced quality of life [6,7]. Specifically, older adults with one or more physical CDs have shown an increased risk of depression and SI compared to those without them [8,9]. Meta-analytic studies comprising 40 and 19 studies revealed that, for individuals with two or more CDs, the risk of depression is two and three times higher, respectively, compared to those with one and no CD [10]. Various chronic conditions, including cancer, stroke, neurological disorders, and metabolic syndrome, have been reported as potential risk factors for depression [11]. Additionally, arthritis, heart disease, and chronic obstructive pulmonary disease have been associated with an increased risk of SI [12,13].
Machine learning (ML) techniques have emerged as efficient tools for predicting health outcomes, including mental health conditions such as depression and SI [14-16]. ML has demonstrated superior performance in predicting mental health outcomes compared to traditional statistical methods [6,17]. ML is increasingly applied to detect, predict, and support the treatment of suicide-related behaviors. Hekler et al. [18] found 4,002 articles that applied ML methods to predict psychological outcomes up to 2021. Long-term conditions such as diabetes, asthma, epilepsy, and chronic pain have been incorporated as one of the predictors of depression using ML models [19]. However, regarding SI prediction, only one ML-based study has included CDs as one of the 26 predictors, and no previous ML-based study has specifically examined the contribution of CDs in predicting SI [20]. Given the association between CDs and mental health outcomes and the role of CDs in increasing vulnerability to mental disorders through disease related distress, functional impairments, and medication adherence [11,12,14-16], incorporating CDs into ML-based predictive models may provide additional insights into the risk factors related to depression and SI.
The existing complex ML models are often criticized for not providing well-explained and interpretable predictions, which are essential for decision-making in healthcare [21]. Explainable artificial intelligence methods have been introduced to help make complex ML-based results more understandable for humans. The ranking of feature importance and Shapley Additive explanation (SHAP) have recently been adopted to enhance explainability [22].
Therefore, we developed predictive models focusing on CDs to identify depression and SI in community-dwelling older adults using five popular ML methods. Additionally, we assessed the independent predictive ability of the number and specific types of CDs by ranking feature importance and using SHAP results to predict depression and SI among older South Korean adults under the control of depressive symptoms.
METHODS
Data source
This study utilized cross-sectional survey data from the Korea National Health and Nutrition Examination Survey (KNHANES), conducted by the Korea Centers for Disease Control and Prevention. The KNHANES produced nationally representative data and comprised health interviews, health behavior surveys, nutritional surveys, and health examinations. To select a representative sample, a multistage cluster probability design based on administrative regions was adopted. We used the KNHANES data from 2013, 2015, 2017, and 2019, which exclusively maintained a consistent questionnaire about SI, asking, “Have you seriously considered suicide in the past year?”
Study population
Initially, 6,329 participants aged 65 and older who participated in 2013, 2015, 2017, and 2019 were selected. Subsequently, 863 individuals with missing values for depression and SI and 47 individuals with missing values for predictors such as cohabitation status, perceived daily stress, hypertension, and lifetime general cigarette use were excluded. Finally, data from 5,419 individuals were used for the analysis (Figure 1). This study was approved by the Korea University Institutional Review Board (KUIRB-2023-0197-01).
Dependent variables
Depression was employed as one of the dependent variables, measured using the question, “Have you experienced feeling sad or hopeless for 2 weeks or more continuously in the past year to the extent that it affected your daily life?” The depression group (n=825, 15.2%) and non-depression group (n=4,594, 84.8%) were assigned based on whether the responses were affirmative or negative.
SI experience served as another dependent variable, measured by the question, “Have you seriously considered suicide in the past year?” The SI (n=396, 7.3%) and non-SI (n=5,023, 92.7%) groups were allocated based on their responses to the questions on SI (Figure 1).
Predictive variables
Ten potential predictors were chosen from sociodemographic, socioeconomic, mental health, and health-related behavioral factors. The three sociodemographic factors were age, sex, and living arrangements (living alone vs. living with others). Three socioeconomic factors were education (less than elementary school, middle school, high school, college, or higher), household income (low, middle-low, middle-high, or high), and employment status. The two mental health factors were depression (experiencing depression for 2 or more consecutive weeks) and perceived daily stress (hardly noticeable, feeling a little bit, feeling a lot, or feeling very much). In addition, the two health-related behavioral factors were alcohol consumption (frequency of alcohol consumption in the past year) and lifetime cigarette smoking. Depression status was only used as a potential predictor of SI.
The CD statuses were determined based on self-reported answers about having specific CDs or having been diagnosed with specific CDs by a physician [23]. The originally chosen 25 CDs from KNHANES data were collapsed into 17 types, including hypertension, dyslipidemia, stroke, myocardial infarction, angina pectoris, osteoarthritis, rheumatoid arthritis, pulmonary tuberculosis, asthma, kidney failure, atopic dermatitis, diabetes, thyroid disease, hepatitis B, hepatitis C, liver cirrhosis, and cancers (stomach, liver, colorectal, breast, cervical, lung, and thyroid cancers, etc.).
The number of CDs among the 17 CDs, categorized as none, one, two, three, and four or more CDs, was included in Model 1 to predict depression and SI. In total, 13 and 14 predictors were used to predict depression and SI, respectively, including four dummy variables, based on the number of CDs and nine other predictors. Model 2 included 17 specific CDs and 27 and 26 predictor variables were used to predict SI and depression, respectively.
ML methods
For the prediction models, we employed logistic regression (LR) with ridge regularization and a support vector machine (SVM) with a linear kernel. Additionally, three tree-based and ensemble models, decision tree (DT), random forest (RF), and extreme gradient boosting (XGB) were employed.
The prediction models were developed according to the following procedure. The entire dataset was randomly split into training and test sets at a ratio of 8:2. A 10-fold cross-validation method was used to prevent overfitting of the prediction models. To avoid duplicates of the same observations, the training set and validation set were divided, and synthetic minority oversampling technique (SMOTE) was applied exclusively to the training set [24]. The class imbalance problem occurs when the size of the majority (negative class) is larger than that of the minority (positive class) [25]. Given the rarity of the minority class examples, which can impact ML prediction performance, SMOTE was applied to balance the datasets in the training set [26]. SMOTE is an oversampling technique that fills the gaps between minority class examples by creating “synthetic” examples, and it has a lower risk of overfitting compared to random oversampling, which simply duplicates minority class examples [27,28].
The optimal parameters for each ML method were tuned using a grid search. The best model was selected based on the area under the curve (AUC) value. Performance with an AUC below 0.6 was interpreted as poor, from 0.6 to 0.75 was moderate, from 0.75 to 0.9 was good, and above 0.9 was excellent [29]. In addition, model performance was evaluated using accuracy, sensitivity (recall), specificity, precision (or positive predictive value, PPV), Matthew’s correlation coefficient (MCC) and F1-score. For the appropriate evaluation of imbalanced data, balanced accuracy (BA), recommended sensitivity and specificity averages, and precision-recall AUC (PR-AUC) were calculated [30].
To assess the importance of the predictor variables, the permutation feature import (PFI) method was consistently applied to all the ML models. The PFI method involves randomly shuffling a specific value, breaking the relationship between the features and the target, and defining the decrease in model performance scores [31].
The SHAP value was utilized to interpret the impact of features on the predicted value, and the contribution of each feature was calculated and arranged by the magnitude of contribution [32]. The x-axis represents the SHAP value, with features that are marked with a red dot to the right of the x-axis indicating a positive impact on model predictions, while decreasing features are colored blue [22,32]. All ML prediction models were analyzed using Python (version 3.7.10; Python Software Foundation).
Basic statistical analysis was performed to assess the bivariate association between depression, SI, and predictors using the chi-square test in SAS 9.4 (SAS Institute). Statistical significance was set at p<0.05.
RESULTS
Characteristics of participants
The participants comprised 43.2% males and 56.8% females with an average age of 72.7 years (standard deviation: 5.0). Among the participants, 15.3% reported experiencing depression, and 7.3% reported a history of SI. Specifically, 10.8% of males and 18.6% of females reported experiencing depression, whereas 6.7% of males and 7.8% of females reported experiencing SI.
Regarding the number of CDs from the 17 CD types, the percentages of participants who had none, one, two, three, or four or more CDs were 22.5%, 33.4%, 27.8%, 13.0%, and 3.3%, respectively. The presence of both depression and SI was significantly and positively associated with the number of CDs, indicating that participants with more CDs had higher percentages of depression and SI (Supplementary Table 1).
Among the individual CDs, dyslipidemia, osteoarthritis, rheumatoid arthritis, asthma, and hepatitis C were significantly associated with depression (p<0.05). SI was associated with osteoarthritis, asthma, atopic dermatitis, and hepatitis B and C (p<0.05) (Supplementary Table 2).
Performance analysis for Model 1
In Model 1, which included the number of CDs as the main dependent variable, the prediction of depression using five different algorithms showed moderate to good performance, with an AUC ranging from 0.726 (XGB) to 0.772 (DT) and a PR-AUC ranging from 0.358 (SVM) to 0.391 (DT) (Table 1). The performances of five models in terms of AUC did not significantly differ. Sensitivity ranged from 0.394 (LR) to 0.618 (SVM), indicating that the algorithms accurately identified approximately 39.4% to 61.8% of cases with depression. The PPV ranged from 0.290 (XGB) to 0.471 (LR), resulting in a 1.91 to 3.1 times increase in predictive power compared with the baseline rate of 15.2% for depression. MCC ranged from 0.245 (RF) to 0.341 (DT), indicating correlation between predicted and actual depression cases; the F1-score ranged from 0.385 (XGB) to 0.680 (RF).
To predict SI in Model 1, all five algorithms showed similarly good performance, with AUC ranging from 0.754 (XGB) to 0.793 (DT) and PR-AUC ranging from 0.179 (SVM) to 0.294 (DT) (Table 1). Sensitivity ranged from 0.380 (RF) to 0.582 (LR), indicating that the algorithms accurately identified approximately 38.0% to 58.2% of cases with SI. The PPV ranged from 0.184 (LR) to 0.317 (DT), resulting in a 2.52 to 4.34 times increase in predictive power compared with the baseline rate of 7.3% for SI. MCC ranged from 0.213 (XGB) to 0.274 (DT), indicating correlation between predicted and actual SI cases; the F1-score ranged from 0.272 (XGB) to 0.327 (DT).
Feature importance and interpretability of depression and suicide ideation risk prediction
To predict depression in Model 1, the top five important variables and their respective frequencies in parentheses across the five models were stress (5), education (4), income (3), one CD (3), and age (3) (Supplementary Table 3). The rankings of the CDs ranged from 2nd (LR) to 10th (DT) for one CD, 6th (SVM) to 11th (RF) for two CDs, 9th (LR) to 13th (DT and XGB) for three CDs, and 10th (XGB) to 13th (RF) for four or more CDs.
To predict SI in Model 1, the top five important variables and their respective frequencies in parentheses across the five models were stress (5), depression (5), income (4), one CD (4), age (3), and education (2) (Supplementary Table 3). The rankings of the CDs ranged from 3rd (SVM) to 9th (DT) for one CD, 6th (SVM) to 14th (DT) for two CDs, 10th (LR) to 13th (RT) for three CDs, and 12th (LR) to 14th (RF and XGB) for four or more CDs.
The SHAP summary plots leverage individualized feature attributions to convey all aspects of a feature’s importance while remaining visually concise (Figure 2) [32,33]. Figure 2 illustrates that higher stress levels positively contribute to increased depression and SI. In Figure 2A, the variables that correlated positively with depression included stress, age, living arrangements, one CD, sex, two CDs, three CDs, lifetime cigarette smoking, and four CDs. In Figure 2B, the variables with a positive correlation with SI include stress, depression, age, lifetime cigarette smoking, living arrangements, three CDs, work status, two CDs, and four or more CDs.
Performance analysis for Model 2
For the prediction of depression in Model 2, with 17 specific types of CDs, the five ML models showed moderate to good performance, with AUC ranging from 0.704 (XGB) to 0.768 (DT) and PR-AUC ranging from 0.329 (XGB) to 0.378 (DT) (Table 2). The sensitivity across the five algorithms ranged from 0.430 (DT) to 0.588 (LR), indicating that the algorithms accurately identified approximately 43.0% to 58.8% of depression cases. PPV ranged from 0.280 (XGB) to 0.452 (DT), resulting in a 1.84 to 2.97 times increase in the predictive power for depression. Regarding depression prediction in Model 2, the DT model exhibited better performance in predicting depression than did the other models, with the highest values for accuracy, BA, specificity, PPV, MCC, F1-score, and PR-AUC at 0.834, 0.668, 0.906, 0.452, 0.344, 0.441, and 0.378, respectively. The LR model exhibited the highest sensitivity (0.588). Based on the model performance metrics, the DT model was selected as the best-performing model for predicting depression in Model 2.
For the prediction of SI in Model 2, the AUC ranged from 0.750 (SVM) to 0.785 (DT) and the PR-AUC ranged from 0.212 (RF) to 0.277 (DT), demonstrating good performance (Table 2). The sensitivity across the five algorithms ranged from 0.418 (RF) to 0.595 (LR), indicating that the algorithms accurately identified approximately 41.8% to 59.5% of SI cases. The PPV ranged from 0.187 (SVM) to 0.296 (DT), resulting in a 2.56 to 4.05 times increase in predictive power for SI. To predict SI in Model 2, the DT model showed superior performance compared with the other models, with the highest values for accuracy, specificity, PPV, MCC, F1-score and PR-AUC of 0.880, 0.912, 0.296, 0.310, 0.363 and 0.277, respectively. The LR model exhibited the highest BA (0.701) and sensitivity (0.595). Based on the model performance metrics, the DT model was selected as the best-performing model for predicting the SI in Model 2.
Feature importance and interpretability of types of CDs in Model 2
To provide a clear explanation of the influence of CD type on prediction in Model 2, we present the ranking of 17 CD types. For the top five important variables among both CDs and other predictors of depression, the order of predictors was similar to that in Model 1. The variables with permutation importance values in parentheses are stress (0.219), household income (0.044), age (0.007), osteoarthritis (0.005), and sex (0.004). Among CD types, the top five CDs predicting depression were osteoarthritis (0.005), myocardial infarction (0.002), diabetes (0.001), asthma (0.001), and stroke (0.001) (Table 3).
Similarly, for the top five important SI variables, the rankings were comparable to those of Model 1. The variables with permutation importance values in parentheses are stress (0.227), depression (0.139), income (0.072), age (0.063), and alcohol consumption status (0.023). Among the types of CDs, the top five CDs predicting SI most effectively were stroke (0.011), hypertension (0.010), asthma (0.009), myocardial infarction (0.009), and rheumatoid arthritis (0.008) (Table 3).
DISCUSSION
This study compared the applicability of five ML predictive models, namely DT, RF, XGB, LR, and SVM, in predicting depression and SI among older Korean adults from a population-based survey. Prediction models for depression and SI were successfully constructed, and the model revealed that the number of CDs and CD types contributed to the risk of depression and SI in late life. Despite the importance of CDs in old age, ML prediction studies on SI are scarce. To the best of our knowledge, this is the first study to emphasize the impact of the number and types of CDs on the prediction of SI using ML models among older adults.
The five ML prediction models performed well, with AUC values ranging up to 0.772 and 0.793 for depression and SI, respectively, in the model including the number of CDs (Model 1), and with AUC values ranging up to 0.768 and 0.785 for depression and SI, respectively, in the model including types of CDs (Model 2). The model performance in this study is comparable to that in previous studies. A meta-study reported an average AUC of 0.78 in the prediction of depression, encompassing 19 ML studies of adequate quality out of the initial 744 studies published from 2010 to 2022. Of the 19 studies, 13 included CD variables in predicting depression [19]. Bernert et al. [14] performed a meta-analysis that included 87 studies on suicide and reported AUC values ranging from 0.61 to 0.97 among 19 ML studies predicting SI. However, none of the studies included CDs predicting SI. To predict SI, we found only one study considering CD; Na et al. [20] established a prediction model of SI with the experience of CD diagnosis as a predictor and reported an AUC value of 0.87 in South Korean population.
In this study, DT showed equal or even better performance than the other ML methods, although ensemble methods such as RF or XGB were expected to provide superior performance compared to a single DT model. Mohammadian et al.’s results [34] showing that DT had a greater AUC than LR and RF in the training sample are in concordance with this study.
Numerous CDs demonstrated a positive increase in the prediction of both depression and SI, as assessed using SHAP values. Having at least one CD contributed positively to predicting depression, whereas having at least two CDs showed a positive increase in the prediction of SI in this study. Hwang et al. [35 ]used traditional analytic methods to highlight a significant relationship between having two or more CDs and depression among Korean older adults. Compared to the population without CDs, men with CDs exhibited a 2.1-fold higher risk of depression, while women with CDs had a 1.9-fold higher risk [35]. Similar results were observed in the SI model, in which Smith et al. [36] reported significantly higher odds of SI associated with having one, two, three, and four or more CDs compared to having no CD. However, we did not find any positive correlation between one CD and SI by SHAP value analysis. The distinction between the risks posed by a single CD and multiple CDs in predicting SI remains unclear and warrants further research. A path analysis conducted by Zhu et al. [37] indicated that CDs directly or indirectly influence SI in older adults, either through direct pathways or via psychological distress. This suggests that the relationship between CD and SI is complex and dependent on the physical, emotional, and social contexts of older adults [37].
Among the 17 CD types, the top five CD predictors of depression were osteoarthritis, myocardial infarction, diabetes, asthma, and stroke. Consistent with our study, Jiang et al. [38] reported that individuals with a history of stroke had a 2.81 times higher probability of developing depression compared to those without CDs, and the coexistence of arthritis and stroke significantly elevated the risk of depression. Additionally, asthma showed a strong association with depression [38]. For the prediction of SI, the top five CD predictors were stroke, hypertension, asthma, myocardial infarction, and rheumatoid arthritis. This is consistent with Smith et al. [36], which associated arthritis, angina, asthma, and stroke with significantly higher odds for SI. Additionally, Joshi et al. [39] supported our findings, indicating that patients with cardiovascular disease, stroke, ischemic heart disease, and diabetes were more likely to develop SI. Furthermore, arthritis, angina, asthma, and stroke were reported to increase the likelihood of SI significantly [36].
In both depression and SI prediction models, stress was consistently identified as the most important predictor, followed by depression in the SI prediction model. Stress contributes to depression, a major factor in promoting SI in older adults, and the interaction between high levels of stress and depression increases the likelihood of SI significantly [40]. Furthermore, experiences of psychological stress are linked to the onset of depression in the general population, and individuals with depression are more vulnerable to adverse health impacts from stress [41,42]. Additionally, a statistically significant correlation was found between depression and SI [43].
This study has several limitations. First, the cross-sectional design inherently limits the ability of ML analysis to predict future depression and SI. Schafer et al. [44] argued that ML models using longitudinal data performed better in classifying suicide-related outcomes. Second, the use of self-reported single-item questions may introduce recall bias and potential misclassification [45]. Third, our study did not consider the severity of CDs, which can impact the accuracy level, as prior research has shown that both the number and severity of CDs influence depression [46]. Finally, because of the predetermined scope of the existing assessments, we could not perform comprehensive analyses to explore the contribution of realistic combinations of CDs and other health-related factors.
This study evaluated how CDs contribute to the prediction of depression and SI using various ML models in a representative sample of the Korean older adult population. While previous studies have included CDs as secondary predictors, our findings highlight their substantial contribution to predictive accuracy, particularly among older adults with multimorbidity. The number and specific types of CDs contribute to the prediction of depression and SI among older Korean adults. These results may help enhance the prevention, early detection, and treatment strategies for older adults with multimorbidity that tends to involve both CD conditions and mental disorders.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Supplement1
Print
