INTRODUCTION
Suicide is a serious global public health issue. In 2019, more than one in every 100 deaths worldwide (1.3%) resulted from suicide [1]. The rate of suicide attempts far exceeds the rate of completed suicides [2]. However, there are limited treatment options available for patients facing imminent suicide risks. Traditional approaches to manage imminent suicide risk include psychiatric hospitalization and frequent monitoring (e.g., every 15 minutes) by healthcare professionals. These interventions are often inefficient and time-consuming, and do not completely prevent suicide attempts, even during hospitalization [3].
Wearable device monitoring can serve as an effective tool to identify patients in an at-risk state. Data collected in real time from the device can detect an at-risk state for suicide and potentially prevent suicidal behavior. The wearable device can gather a variety of data, including behavior patterns, mood, and physiological data. Specifically, physiological data such as heart rate (HR) or heart rate variability (HRV) can indicate unconscious or unresponsive states. Given that HRV is a marker of parasympathetic activity associated with emotion dysregulation, a study has shown that individuals with a history of suicide attempt have lower HRV measures compared to those without such history [4]. We previously reported that resting HR and HRV measures are significantly associated with recent suicide risk [5] in cross-sectional studies.
Although wearable devices have gained attention as tools for suicide risk prediction [6], research in this area remains limited. Small sample sizes [7], biased participant selection [8], and study designs [9] impede the detection of significant signals in wearable device data. Earlier studies largely demonstrated the potential of wearable devices for evaluating suicide risk, yet conclusive evidence regarding their predictive value remains scarce [10]. Particularly, while a wearable device can collect a variety of information, it is still uncertain whether this data can be effectively integrated to enhance suicide risk prediction.
In this study, we explored whether patients at risk for suicide could be identified using the diverse data gathered from a commercially available wearable device. Given that wearable devices can provide varied information including an individual’s mood, behavioral data, and physiological states in real-time, we hypothesized that this data could be used to predict suicide risk. Furthermore, we utilized commercially available wearable devices due to their ease of implementation in an outpatient setting.
METHODS
Study participants
This study was a secondary analysis utilizing data collected to construct a predictive model for identifying individuals with clinically significant depression. A total of 59 participants (39 with major depressive episodes and 20 age- and sex-matched healthy controls) aged between 20 and 55 years were recruited from the outpatient clinic of the Department of Psychiatry at Samsung Medical Center between December 2019 and October 2020. The patient group consisted of individuals diagnosed with a major depressive episode according to the Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 criteria. Those experiencing significant depressive symptoms, defined as scoring 3 (mildly ill) or higher on the Clinical Global Impression of Severity scale [11] at the time of assessment and who agreed to wear the device for two months, were included in the study. We excluded individuals with schizophrenia, other psychotic disorders, alcohol use disorders, organic mental disorders, mental retardation, and neurological illnesses, including epilepsy. Participants with acute suicide risk were also excluded. Age- and sex-matched healthy controls were recruited through advertisements.
Of the patients included in the study, 20 were diagnosed with major depressive disorder and 19 with bipolar disorder, defined by a previous history of mania/hypomania. Participants in the patient group received standard psychiatric pharmacotherapy for their conditions, as determined by their clinician.
Initially, board-certified psychiatrists assessed the participants’ psychiatric and medical histories to confirm their eligibility. A trained psychologist, blinded to the psychiatrists’ evaluations, independently assessed the participants’ psychiatric diagnoses and current mood states. The psychiatric diagnoses were based on DSM-5 criteria [12]. The following instruments were employed to assess participants’ mood symptoms: the Montgomery-Asberg Depression Rating Scale (MADRS) [13], the Hamilton Depression Rating Scale (HAMD) [14], the Bipolar Depression Rating Scale (BDRS) [15], and the Young Mania Rating Scale (YMRS) [16]. The MADRS and HAMD are clinician-administered scales extensively used to quantitatively assess depression severity. The BDRS is another clinician-administered scale for measuring depression, with a specific focus on symptoms prevalent in bipolar depression. The YMRS quantifies manic symptoms.
All study procedures received approval from the Institutional Review Board of the Samsung Medical Center (IRB no. 2019-12-043). Written informed consent was obtained from all participants following a full explanation of the study. The consent form also described the purpose of the secondary analysis. All data were anonymized before analysis to protect participants’ privacy.
Wearable device monitoring
We used the commercially available Galaxy Watch Active2 to collect both active and passive data from participants over two months. Participants were required to wear the device on their wrist continuously for two months, except during showering, when we advised charging the device. Participants were incentivized based on their compliance with the device-wearing schedule during the study.
The Galaxy Watch Active2, operating on Tizen OS, features an accelerometer, barometer, gyro sensor, light sensor, optical HR sensor, and electrical heart sensor. Its photoplethysmography or photoplethysmogram sensor incorporates eight photodiodes [17]. Available in various sizes and editions, our study utilized a 40 mm version without long-term evolution technology. The device captures HR and HRV data, recording HR every minute. Previous research has confirmed the Galaxy Watch Active2’s accuracy in measuring HR and HRV [18]. The Galaxy Watch Active2 has been employed in numerous studies [18-23] and has demonstrated acceptable accuracy in detecting physiological changes, comparable to other commercially available wearable devices [20].
Using the wearable device, we collected daily behavioral data including activity level (step count, mobility index, running and walking time, and exercising time), sleep time, resting time, and wear-free time (the amount of time participants removed the device daily). The movement index represented the cumulative movement per minute. Sleep time, measured using the accelerometer, provided results comparable to those obtained from polysomnography [19].
The wearable device incorporates an automated classification algorithm that determines if an individual is experiencing acute stress by monitoring HR changes during the Trier social stress test [24]. Individuals whose HR changes suggest a greater than 75% likelihood of acute stress are classified as being in a high-stress state. If the likelihood is lower than 25%, they are classified as being in a low-stress state. A state falling between these two is classified as moderatestress. The device automatically calculates the time spent within 24 hours in each of these stress states.
We incorporated ecological momentary assessment (EMA) to evaluate the participants’ mood states on the wearable device. We inquired about four different emotional states twice daily using the statements: “I am depressed,” “I am anxious,” “I am up and hyper,” and “I am stressed out.” When displayed on the device screen, participants responded on a scale from 0 (none) to 4 (serious) by touching the screen.
Table 1 outlines the variables measured using the wearable devices. The dataset for building the predictive model was segmented into three categories: daily behavioral data, EMA data, and physiological data.
Follow-up evaluation by clinicians
Participants returned to the clinic for follow-up evaluations at weeks 2, 4, and 8. During these visits, clinicians assessed participant compliance and experiences, and also evaluated MADRS, HAMD, BDRS, and YMRS.
Suicide risk evaluation
The HAMD suicide item score, HAMD-3, was employed as the primary outcome measure for assessing imminent suicide risk, as reported in a prior study [25]. The HAMD-3 is extensively used to quantitatively evaluate the risk of suicide [26-29]. The HAMD-3 is evaluated by clinicians who assess suicidal ideation and behavior to characterize the suicide risk state over the previous two weeks on a scale from 0 to 4, with 0 indicating no suicidal ideation and 4 indicating a serious attempt. In subsequent analyses, HAMD-3 scores ≥1 were classified as indicative of imminent suicide risk. As previously noted, the HAMD-3 assessments were conducted at weeks 0 (baseline), 2, 4, and 8. In addition, both item number 10 of the MADRS and item number 13 of the BDRS, which also evaluate suicide risk over the previous two weeks, were utilized as markers for imminent suicide risk during the analyses.
Statistical analysis
Prior to the machine learning analysis, we compared the differences between the patient group and healthy controls regarding collected variables. Moreover, we assessed the disparities between participants with bipolar and unipolar depression within the patient group to confirm their suitability as a uniform cohort.
Figure 1 briefly summarizes the schematic data analysis pipeline, detailing how the collected datasets are processed throughout, from preprocessing (including data synchronization, normalization, and feature selection) to prediction through machine-learning and deep-learning models, utilizing several libraries, such as scikit-learn and the PyTorch framework, and implemented using the Python programming language.
Following prior studies [30], we included only days when the participants wore the wearable device for more than 60% of the day. Subsequently, we merged the raw data into two datasets with different time windows, which served as inputs for the deep-learning models. These models predicted the next status by analyzing the previous status during the time window. The model applied to EMA scores utilized a three-day dataset, while the input dataset for the HAMD score model spanned 14 days.
Pre-processing
As previously mentioned, the dataset for the predictive model was divided into three categories: daily behavioral data, EMA data, and physiological data. Due to varying timeframes and characteristics of each dataset, it was necessary to synchronize and normalize the data, including HR, EMA, and physiological data. We synchronized all data on a daily basis. Each dataset was collected within different timeframes; for example, behavioral data was measured daily, and EMA questions were posed twice per day. However, inputs for machine-learning models must be organized into identical timeframes. Each dataset was either extrapolated or interpolated to align with other datasets. For example, the model input was synchronized with the dataset measured twice daily. HAMD, typically observed biweekly, was interpolated to twice daily. The dataset was normalized to rescale values to a range from 0 to 1.
To manage missing values and outliers, we removed implausible HR values, specifically those outside typical human HR ranges. We assessed the number of rows in raw data during the synchronization target period and extracted data only from sections that provided sufficient data, thereby excluding missing values.
After pre-processing, we constructed a correlation matrix that included all variables and group membership.
Generation of prediction model
We utilized machine learning methods to predict suicide risk using the dataset. Specifically, we developed two distinct machine-learning models for assessing suicide risk and compared their effectiveness. In the single-step model, we input all available data into one model, treating imminent suicide risk as a dependent variable. In the three-step model, we emulated the classic clinical approach to suicide prediction: 1) identifying a high-risk group based on psychiatric diagnoses with depression being a significant factor in evaluating suicide risk [31], 2) classifying those in an at-risk state based on mood symptoms, particularly current depression, using EMA data as the dependent variable in the second step, and 3) identifying individuals with imminent suicide risk.
The prediction model inputs were normalized to a range from 0 to 1. We utilized an 8:2 ratio for training and validation. Accuracy was determined using root mean square error (RMSE).
The prediction model was developed using a machine-learning model through H2O AutoML and a deep-learning model constructed via PyTorch. AutoML facilitated the identification of optimal machine-learning models from a selection including generalized linear models (GLM and Deep-Learning), distributed random forest (DRF), extreme gradient boosting (XGBoost), gradient boosting machine (GBM), and StackedEnsemble, all operated using H2O AutoML of R Studio [32]. AutoML sought both the best machine-learning model and the ideal hyperparameters to maximize performance. Figure 1 depicts the AutoML procedure, which is iterated automatically without human input. The limits imposed by hyperparameters and the network model restrict the AutoML’s deep learning. Consequently, a custom deep-learning approach was created using PyTorch. Given these limitations, tailored neural networks were employed on the same dataset used for AutoML.
RESULTS
Overall, participants maintained a retention rate of 75.7% throughout the experimental period. The patient group used the device for 75.7% of the study duration, while the healthy control group used it for 75.9%. No significant differences in retention rates were found between the patient and healthy control groups.
Table 2 presents the basic characteristics and baseline mood symptoms of the participants. The patient group exhibited increased mood symptoms across all assessed categories, including MADRS, HAMD, BDRS, and YMRS scores. Nevertheless, no significant differences were observed between the unipolar and bipolar depression groups (Supplementary Table 1). Regarding imminent suicide risk (HAMD-3 ≥1), 35 (89.7%) of the patient group faced imminent suicide risk at baseline (T0), 31 (79.5%) at T1, 30 (76.9%) at T2, and 27 (69.2%) at T3.
In the EMA data, the patient group exhibited more pronounced fluctuations in their responses over time compared to healthy controls (Supplementary Figure 1). Variations in the healthy controls’ responses were minimal. There was a high correlation between EMA data and imminent suicide risk, with correlations between 0.8 and 0.9 observed for depressed mood (Q1), anxious mood (Q2), and subjective distress (Q4) (Supplementary Figure 2). Elated mood (Q3) showed low correlation with other EMA questions and diagnoses. Therefore, we summed these answers, standardized scores between 0 and 100 (with higher scores indicating more severe daily reported symptoms), categorized them as negative mood, and utilized them in the second step as a dependent variable. Elated mood (Q3) was excluded from further analysis.
Prior to the machine-learning analyses, we explored correlations between each variable and diagnosis. In the correlation matrix that included variables in the analysis, imminent suicide risk demonstrated weak yet significant associations with group membership (patient group vs. healthy control), MADRS total score, HAMD total score, BDRS total score, YMRS total score, anxious mood, mobility index, mean HR, and standard deviation of HR; also noted were the standard deviation of N-N interval, total power (Tp), very low frequency, low frequency (Lf), and total successive R-R interval difference during measurement (Tsrd) (Supplementary Figure 3).
When all variables were included in the single-step model via AutoML, the GBM model yielded the most accurate predictions (mean squared error [MSE]=0.25, RMSE=0.50, area under the curve [AUC]=0.88) (Figure 2A). The HAMD total score showed the highest feature importance, followed by diagnoses and high frequency (Hf). Shapley additive explanations (SHAP) indicated diagnosis, physical stress index (Psi), approximate entropy, and HAMD total scores as the variables with the greatest importance in the model. The summary plot highlights that HAMD total and diagnosis played pivotal roles in the model (Figure 3). The three-step model also proved significant in predicting individuals at imminent suicide risk, with the GBM model achieving slightly better accuracy than the single-step model (MSE=0.25, RMSE=0.50, mean absolute error=0.002, AUC=0.89) (Figure 2B and C). Regarding variable importance, normalized Hf, HAMD total score, and Psi emerged as the most crucial features. In the SHAP explanation, Hf, Tsrd, and Lf demonstrated the highest importance. The summary plot revealed that the HAMD total score was the most significant variable in the model (Figure 4).
DISCUSSION
This study aimed to explore how digital phenotypes gathered from a commercially available wearable device can predict imminent suicide risk. Utilizing all available data, we developed a single-step model and a three-step model. Both models were successful, with the three-step model performing slightly better (AUC=0.88 vs. 0.89), demonstrating the wearable device’s potential as a tool for detecting imminent suicide risk.
This is the first study to apply a multi-step model to suicide risk prediction. Our three-step model mirrors the typical suicide risk assessment process in clinical practice. Compared to the single-step model, this multi-step approach may better achieve the practical objectives of suicide risk prediction. In the single-step model, diagnosis and HAMD scores were identified as the most influential variables. However, psychiatric diagnosis and objective mood assessments can only be conducted by trained clinicians and require significant time. In the current clinical setting, the primary challenge in suicide risk prediction lies in discerning imminent risk among a high-risk group defined by diagnosis and severity of depression.
The current mood state, encompassing both objective ratings and EMA data, was a highly significant variable in our predictive model, aligning with findings from previous studies [7,33,34]. However, reliance solely on EMA data for suicide risk assessment is precarious due to the risk of respondent fatigue [35]. As previously mentioned, clinicians’ mood assessments are among the most dependable measures but are extremely time-consuming.
Thus, incorporating diverse datasets is crucial to mitigate the limitations of mood assessments in predicting suicide risk. Notably, in our study, HRV variables were identified as key factors in both suicide risk models. In earlier research utilizing cross-sectional data, HRV variables demonstrated significant correlations with suicide risk [5]. While not as decisive as mood state evaluations, our findings highlight the potential of physiological data collected via wearable devices in monitoring suicide risks. Given that HRV measurements can be reliably captured in real-time by wearable devices, they hold promise for real-time suicide risk assessment.
Although each HRV variable might reflect distinct aspects of autonomic nervous function, it remains uncertain which HRV variable is most effective for suicide risk monitoring. Further research is necessary to elucidate the significance of HRV variables in suicide risk prediction.
Wearable device recently has been utilized in mental health monitoring. Most studies focused on anxiety, depression and stress [36], but limited studies exist on suicide risk monitoring. Several studies have attempted to use mood ratings with physiological data collected via wearable devices in suicide risk monitoring, yielding inconsistent results. Various factors such as sample size, participant ethnicity, and methods of suicide risk assessment could influence these findings. Czyz et al. [7] conducted a longitudinal prognostic study involving patients who presented at an emergency department with recent suicidal ideation or attempts over eight weeks. They assessed suicide risk using the EMA method and discovered that while EMA data were effective in predicting suicide risk, sensor data did not enhance the predictions when combined with EMAs. Kleiman et al. [37] developed a predictive model for suicidal ideation using data from wearable devices and EMA, which included mood-related questions and assessments of suicidal thoughts. The model incorporating physiological data underperformed compared to those using only EMA data. Nevertheless, physiological data enhanced the model fit with EMA, indicating that when interpreted in the context of emotions, physiological data could improve predictions of suicidal ideation. Although they did not develop a predictive model, Sheridan et al. [9] also reported that HRV was significantly associated with suicide risk.
Most previous studies on mood ratings have relied solely on EMA data, i.e., self-rating mood scale ratings [7,9]. However, our study also incorporated objective mood ratings, the gold standard for assessing mood symptoms in patients. In our analysis, objective mood ratings demonstrated a more significant role in predicting suicide risk than EMA data. Although EMA data correlated highly with objective mood ratings, EMA readings from individuals with depression exhibited more frequent fluctuations compared to those of healthy controls [38], as we also observed. Clinician-administered mood ratings may be simpler and better capture the core features of depression symptoms [39]. Moreover, objective mood ratings might reveal different aspects of depression symptoms than self-rating scales [40]. Clinician-assessed suicide ratings also indicate dimensions distinct from self-reported suicide risk [41]. Future research should incorporate both objective mood ratings and EMA data to further investigate the nuances of an individual’s mood symptoms and suicide risk.
Contrary to mood ratings, behavioral data, encompassing activity levels and sleep-related information, did not significantly influence suicide risk prediction. This finding aligns with recent research by Czyz et al. [7] directly observable behaviors such as activity or sleep by others may not reliably indicate imminent suicide risk. Nonetheless, other types of behavioral data, like geolocation or phone usage patterns, could represent a distinct factor in assessing suicide risk.
In our study, we combined two patient groups (unipolar and bipolar depression groups) into a single group after confirming no significant differences in wearable device data and clinical characteristics between them. Although bipolar depression (bipolar disorder, currently in a depressive episode) and unipolar depression (major depressive disorder) are distinct conditions in the DSM system, their diagnostic criteria are identical [42]. Consequently, distinguishing between the two conditions based on clinical features alone is challenging. Digital phenotypes of bipolar and unipolar depression might be indistinguishable during an acute depressive episode.
To successfully implement this predictive algorithm in a real-world clinical setting, active use of the wearable device by participants is crucial. Although no significant group differences were observed, there was a general trend towards greater non-wear time among patients, confirming previous findings of low compliance in individuals with depression [43,44]. To implement the algorithm effectively in real-world settings, specific strategies must be developed to engage and motivate high-risk groups to fully participate.
Limitations
Our study findings should be interpreted within the context of the study. First, the sample size may be inadequate to substantiate the study findings. Specifically, the healthy control sample size was smaller compared to the patient group. Additionally, validation samples were not included. Instead, we included high-risk populations seeking psychiatric treatment. A previous study that failed to detect significant signals [8] involved large (n=2,881), yet low-risk populations. However, it is still difficult to generalize our model due to limited samples and potential bias related to sample size and recruitment pattern. Second, the timeframe for exploring imminent suicide risk may not have been suitable. Prior research [45] indicated that suicidal risk varies over time, even across a few weeks. Therefore, suicide risk in the prior two weeks may not remain stable throughout the observation period, and a more refined evaluation of suicide risk using digital phenotypes might be more suitable. However, two weeks is generally recognized as the most acceptable timeframe for suicide risk evaluation. Third, the timeframe for the behavioral data utilized in our study may not have been appropriate for risk evaluation. Daily data is too general to accurately capture an at-risk state, given that suicide risk continually fluctuates over time. Real-time monitoring is essential to identify moments of suicidality accurately. Future studies should explore digital phenotype data with finer temporal resolution. Fourth, we did not include digital phenotype variables such as geolocation or communication patterns. Incorporating these variables may enhance the prediction of suicide risk. Fifth, generalizing the study findings to different cultural or ethnic groups may be challenging. Behavioral patterns are deeply influenced by cultural backgrounds. Given that most studies on digital phenotypes have been conducted with Caucasian populations, this study contributes additional evidence that could facilitate understanding digital phenotypes across diverse populations. Sixth, as previously stated, individuals with more severe depressive symptoms showed higher non-wear time, indicating challenges in maintaining wearable devices among high-risk populations. Efforts should focus on motivating high-risk groups to participate in studies. Seventh, machine learning analysis has inherent limitations, particularly its dependency on trained data, which could lead to overfitting.
To overcome limitations of our study, rigorous future studies with careful design are necessary. The future study should apply a finer time frame, both in suicide risk evaluation and in passive data monitoring. It would be ideal if we could track suicide risk in real time. Furthermore, diverse digital phenotypes not included in our study, such as geolocation, phone usage patterns, and electrodermal activity, could also provide valuable information. Larger samples from ethnically and culturally diverse backgrounds should also be included in future studies.
Conclusions
Our study demonstrated the potential use of a commercially available wearable device for monitoring suicide risk in an outpatient setting. By utilizing wearable device data collected over a sufficient period and mood ratings, we developed a prediction model applicable to suicide risk monitoring. Although physiological data from the wearable device alone did not predict suicide risk, it showed potential for suicide risk prediction when correlated with daily mood symptoms. Further studies involving larger sample sizes and data with finer time resolution are warranted to generalize these findings.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Supplement1
Print
