Ko, Lim, Park, Kim, Suh, Lee, Lee, Moon, Kim, Park, Shin, and Lee: Subjective Experience on Virtual Reality-Assisted Mental Health Promotion Program

Abstract

Objective

Mental health promotion programs using virtual reality (VR) technology have been developed in various forms. This study aimed to investigate the subjective experience of a VR-assisted mental health promotion program for the community population, which was provided in the form of VR experience on a bus to increase accessibility.

Methods

Ninety-six people participated in this study. The relationship between the subjective experience and mental health states such as depression, anxiety, perceived stress, and quality of life was explored. The subjective experience on depression and stress before and after VR program treatment was compared using the Wilcoxon signed-rank test. The satisfaction with the VR-assisted mental health promotion program was examined after using the VR program.

Results

The VR-assisted mental health promotion program on a bus significantly improved subjective symptoms such as depression (p=0.036) and perceived stress (p=0.010) among all the participants. Among the high-risk group, this VR program significantly relieved subjective depressive feeling score (p=0.033), and subjective stressful feeling score (p=0.035). In contrast, there were no significant changes in subjective depressive feelings (p=0.182) and subjective stressful feelings (p=0.058) among the healthy group. Seventy-two percent of the participants reported a high level of satisfaction, scoring 80 points or more.

Conclusion

The findings of this study suggest that the VR-assisted mental health promotion program may effectively improve the subjective depressive and stressful feelings. The use of VR programs on buses to increase of accessibility for the community could be a useful approach for promoting mental health among the population.

INTRODUCTION

The demand for community-based mental health care has increased over the past several decades. Deinstitutionalization has provoked the necessity for community-based mental health care programs [1]. The reduction of stigma and increased interest in mental health within the community have led to the development of strategies for promoting mental health [2-5]. One study reported that a psychoeducation program on mental health reduced caregiver burden, anxiety, and depression [6]. Additionally, community activity programs such as music classes, exercising, and gardening have been tried in community centers [7]. These programs appeared to be partially effective in reducing anxiety and depression in young adults [7]. One systematic review found that the community-based psychiatric treatment focused on social support improved depressive symptoms and overall psychiatric symptoms [8]. In a recent meta-analysis study on the geriatric population, some psychosocial interventions such as reminiscence therapy, cognitive behavioral therapy (CBT), exercise, and social engagement showed favorable outcomes [9].
Recently, mental health care programs using virtual reality (VR) [10] technology have demonstrated good efficacy in treating psychiatric disorders such as anxiety disorders [11,12], posttraumatic stress disorder and obsessive-compulsive disorders [13], and depressive disorders [14,15]. In addition, VR-assisted program has shown effectiveness in community-based settings [10]. VR programs for elderly community members using a kind of functional game have also proven in reducing depressive or anxiety symptoms [16]. However, because most of the subjects in the community have minimal psychiatric symptoms, the effect of the VR program may not be significant. Performances on VR programs in the community varied when comparing VR performances according to clusters of psychiatric symptoms [17]. Based on the findings on previous studies using VR-assisted programs appear to be potentially effective for promoting mental health in the community.
VR-assisted programs offer various merits compared to other psychosocial interventions. VR technology allows utilization of various simulated environments or situations based on therapeutic goal [18]. It is also effective in inducing fearful or positive emotion through immersive experiences [18-20]. In addition, VR-assisted programs provide an excellent way to effectively serve a large number of community populations. Utilizing the accessibility of VR technology, it is possible to effectively provide psychosocial interventions to people who have difficulties in coming to the hospital. Although mental health promotion programs using VR for the community population are being attempted, there is a lack of studies examining the efficacy of VR programs on a bus to maximize accessibility of VR. Therefore, this study aims to investigate the subjective experience of VR-assisted programs on a bus for mental health promotion within the community population to explore the usefulness or applicability of mobility-based VR program.

METHODS

Subjects

The subjects of this study were individuals who participated in the mobility-based flexible medical service project supported by Busan Metropolitan. Ninety-six subjects participated from September 1, 2021 to December 31, 2021. Only those who agreed to VR training were included. Exclusion criteria were 1) those who have an intellectual disability that makes it difficult to perform VR training, 2) people who cannot perform VR training due to visual impairment, and 3) illiterate people. This study was approved by Institutional Review Board at Pusan National University Hospital (PNUH IRB: No 2203-033-112).

Study design

This study examined the improving effect on subjective depressive symptoms and stressful feelings before and after a VR-assisted mental health promotion program for the community population. The relationship between subjective feelings and clinical variables such as depression, anxiety, stress perception, and quality of life was analyzed at baseline. Furthermore, this study aimed to analyze the difference in the subjective symptom improvement effect according to the severity of symptoms in pregnant women. Demographic factors such as age, gender, education, marital status, and job were assessed. The subjective feelings on depressive symptoms and perceived stress were evaluated by a VR-based survey before and after using the VR program. Depression, anxiety, stress perception, and quality of life were evaluated by the Patient Health Questionnaire-9 (PHQ-9), Generalized Anxiety Disorder-7 (GAD-7), Perceived Stress Scale (PSS), and World Health Organization Quality of Life Scale Abbreviated Version (WHOQOL-BREF), respectively.

VR program

The VR program used in this study (Figure 1) is a simple program consisting of one module among smart mental care VR programs developed to improve mental health and quality of life. Previous studies have reported that the smart mental care VR training program reduced depressive symptoms and anxiety while improving the quality of life [16,17]. In the previous studies, the VR training program comprised three distinct modules (referred to as Module 1, Module 2, and Module 3). The protocol involved a daily training 30 minutes, twice a week, a total duration of 12 weeks. However, the original version of this program necessitated a prolonged training period and multiple repetitions, rendering it less feasible for application in community settings. In order to be used as a community mental health management model, it must be simple and highly accessible. Consequently, Module 1 in this study was chosen due to its suitability for mitigating negative emotions such as depression and anxiety, as well as its relative ease in terms of difficulty. The Module 1 used in this study involves approximately 5 minutes of training. During the VR module, the participants can click a button on the VR controller to hunt only bad birds with protecting good birds. Before and after the training, the participants were asked to score how much they felt depressed and stressed.

Measurements

Survey on subjective feelings

The subjective feelings on depressive symptoms and perceived stress were evaluated by a VR-based survey before and after using the VR program. The subjective depressive feeling was scored on a 6-step Likert method, with 0 point for ‘not at all’ and 100 points for ‘totally so’ in response to the sentence ‘I feel down or depressed’. Similarly, subjective stressful feeling was also scored with 0 point for ‘not at all’ and 100 points for ‘totally so’ in response to the sentence ‘I have a hard time due to various stressor’. The subjective satisfaction after VR training was evaluated on a 6-step Likert method, with 0 point for ‘not satisfied at all’ and 100 points for ‘totally satisfied’ with respect to the sentence ‘This VR training has helped me’.

PHQ-9

The PHQ-9 is a self-report questionnaire used to screen for depression and assess its severity. The PHQ-9 consists of nine items, each with a maximum four-point [21]. A total score of 5–10 suggests mild depression, 11–16 suggests moderate depression, and 16 or higher suggests severe depression. In the Korean version of this scale, the optimal cut-off point for major depression was ≥9 (sensitivity: 88.5%, specificity: 94.7%), and it also had considerable reliability and validity [22].

GAD-7

The GAD-7 is a self-report questionnaire to screen for anxiety and measure its severity [23]. With seven items asking about one’s anxiety or worries over the last two weeks, the total score can range from 0 to 21. The optimal cut-off point for the GAD-diagnosis was found to be ≥10 [23]. The Korean version of this scale, which was used in this study, was also reliable, efficient, and valid [24].

PSS

The PSS, originally developed to assess one’s stress level in people aged 12 and above over the last month [25], is a self-report questionnaire. By summing across total ten items regarding how often they felt a certain way-either negative or positive-in certain situations. The total score can range from 0 to 40, with higher scores suggesting higher levels of perceived stress. This study employed the Korean version of this scale, which also had good reliability and validity [26].

WHOQOL-BREF

The WHOQOL-BREF is a self-report 26-item questionnaire, that accesses four domains of quality of life: physical health, psychological health, social relationships, and environment [27]. The total score can range from 26 to 130, with a high score implying better quality of one’s life. The Korean version of this scale, which was used in this study, also shows great reliability and validity [28].

Statistical analysis

Demographic and clinical variables were presented as mean (standard deviation) or number (proportion) according to the characteristics of the variables. The correlation between subjective feelings and clinical variables such as depression, anxiety, stress perception, and quality of life at baseline was analyzed by Spearman correlation analysis. The subjective feelings on depression and perceived stress were compared before and after using the VR program by Wilcoxon signed-rank test among total participants and the high-risk group, respectively. The cut-off score of PHQ-9 (≥8) for screening the high-risk groups was applied [29]. All data were analyzed using the SPSS 22.0 statistical program (IBM Corp., Armonk, NY, USA) with a significance level of p<0.05.

RESULTS

Demographic and clinical characteristics of the participants

The demographic and clinical characteristics of 96 participants are shown in Table 1. The mean age of the participants was 72.8±17.2 years. Three-quarters of the participants were women. The mean scores for PHQ-9, GAD-7, PSS, and WHOQOL-BREF were 3.4 (±4.4), 1.9 (±3.1), 20.8 (±7.4), and 79.6 (±23.2), respectively. The mean scores for subjective depressive feeling and subjective stressful feeling were 47.1 (±20.3) and 47.3 (±21.0), respectively.

Correlation analysis between clinical characteristics by subjective self-rating scales and subjective feelings during VR training

As shown in Table 2, the score of subjective depressive feeling was positively related to the scores of PHQ-9, GAD-7, and PSS. Similarly, the score of subjective stressful feeling and the scores of PHQ-9, and GAD-7 were positively correlated.

The changes of subjective feelings before and after experience of VR-assisted smart mental care

As shown in Table 3, there was a significant decrease in the score of subjective depressive feeling after the VR-assisted mental health promotion program (Z=-2.092, p=0.036). Similarly, the score of subjective stressful feeling also showed a significant decline (Z=-2.592, p=0.010) following the program.

The effects of smart mental care according to PHQ-9 screening results

We also compared two groups screened by the PHQ-9, and the participants were grouped into two categories: the PHQ-9(+) representing high-risk group and the PHQ-9(-) indicating healthy group (Table 4). In the PHQ-9(+) group, the VR-assisted mental health promotion program significantly relieved subjective depressive feeling score (Z=-2.126, p=0.033), and subjective stressful feeling score (Z=-2.111, p=0.035). However, the PHQ-9(-) group showed no significant changes after the VR-assisted mental health program (subjective depressive feeling score Z=-1.335, p=0.182; subjective stressful feeling score Z=-1.895, p=0.058).

Satisfaction of VR training

The mean score of the subjective satisfaction was 74.2 (±19.2) as shown in Table 1. By categorical analysis, 13.5% (13) of the subjects reported 100 points, representing total satisfaction. Fifty-eight percent (56) scored 80 points, thus 71.8% of the subjects showed a high level of satisfaction with 80 points or more. Twenty percent (19) reported 60 points, while 8.3% (8) responded with less than 40 points. The proportion of satisfaction level is shown in Figure 2.

DISCUSSION

This study found that the VR-assisted mental health promotion program might be effectively and satisfactorily employed for the community population using a bus. The results in this study were similar to those of other mental health promoting programs using VR for psychiatric disorders [30-32]. VR technology is becoming an effective approach in the psychiatric field due to its great reproducibility. In a previous systematic review, VR had almost similar outcomes as conventional CBT [18] and, when it is repeated, it showed much better results [18]. In this study, because only one training session was performed, it is challenging to fully verify the effect of this VR program. However, considering the positive changes induced by a single training experience, it is reasonable to expect better effects with repeated application. Some studies have reported VR training difficulties or discomforts, such as dizziness, motion sickness or headache [19,20]. Fortunately, VR training in this study was safely conducted without the discontinuation due to obvious discomfort during VR training. In most cases (72%), participants reported satisfaction level of 80 points or more. Considering the positive subjective experience, VR-based training can be useful for promoting mental health in the community.
Most conventional psychosocial programs are typically implemented in hospitals, but VR-assisted programs have characteristics that allow more diverse implementations, even at home [33]. Especially, the VR program in this study was conducted on a bus to increase accessibility. Nevertheless, the findings showing the improvement of subjective feelings suggest that VR-assisted programs might be clinically useful in terms of accessibility. Mobility-based VR programs could be an active way to manage and promote the mental health of those who have difficulties in visiting hospitals or healthcare centers [34]. Research results on the usefulness of mobility-based services with enhanced accessibility need to be compared to the existing services and accumulated. Additionally, because of insufficient space on a bus and the vibrations that may occur during moving, there might be some difficulties to provide full medical services. However, it is expected that it can be a good alternative to effectively manage the health of subjects who have difficulty visiting hospitals frequently, such as the elderly.
Meanwhile, most of the subjects who participated in this study were elderly individuals. Due to the characteristics of the project based on community health car, the participation of elderly subjects was expected although there was no age restriction. Elderly subjects usually have difficulties in adapting to novel techniques, which can be an obstacle to implement VR technology as a mental health-caring intervention for this population [35,36]. However, this study showed that VR can be successfully adopted to the older population. In a recent study, VR was used to train balance and prevent falls in isolated older adults. There was a significant difference between the VR group and the control group in open eyes balance score [37]. Another study using VR for the people with dementia reported significant improvements in pleasure and alertness after VR-exposure [38]. Likewise, interventions using VR technology are being tried for elderly subjects, and positive results are being reported [33]. It will be a more useful approach if experiences for safe and effective use in elderly subjects are accumulated.
This study revealed positive correlations between the subjective depressive feeling and the scores of PHQ-9, GAD-7, and PSS. Also, the subjective stressful feeling was positively correlated with the scores of PHQ-9 and GAD-7. In addition, this study demonstrated the changes in subjective feelings about stress and depressive mood after using a VR program for 96 participants (Table 1). Specifically, the PHQ-9(+) group exhibited significant improvements in subjective feelings after VR training. The results of this study suggest that VR may be more effective for the high-risk group for depression than for healthy subjects. However, since the subjects in the community are relatively healthy in terms of depression and stress, the change in depression due to VR training is inevitably small, which a larger number of subjects is required to establish a statistically significant effect.
There are some limitations to this study. Firstly, all of the questionnaires were self-reported assessments. In the future, more precise assessments using clinician-administered scales need to be performed. Secondly, this study did not fully verify the effect of a single-session VR program. Only one session of VR training was carried out for each participant as part of a preliminary study. Additional research is needed to ensure the validity of a single-session program. Positive subjective feelings even with one session might imply much better outcomes after repeated interventions. Thirdly, the number of the subjects who participated in this study was too small to generalize the results. The number of the subjects assigned according to the PHQ-9 screening results also seemed to be insufficient to analyze the outcomes of each group. In the future, therefore, a large-scale study needs to be conducted. Fourthly, this study was exploratory and compared various variables but did not adjust the familywise errors due to multiple comparisons. For this reason, caution is needed in interpreting the results. However, the results of this study might provide useful information for conducting future research. Lastly, this study did not use the active control and randomization to compare the efficacy of the VR program. In the future, randomized controlled trial needs to be performed to obtain strong evidence.
In conclusion, the VR-assisted mental health promotion program for the community population appears to be effective in alleviating subjective depressive and stressful feelings. Furthermore, mobility-based approach using VR technology is promising in terms of accessibility. Especially, the VR-assisted program seems to be effective for the subjects complaining the depressive symptoms with PHQ-9(+) results. These findings suggests that the effects of VR program for the community population can differ depending on the severity of psychiatric symptoms. Consequently, this study highlights the potential usefulness of mobility-based VR program. In future studies, a large-scale, randomized controlled study on the efficacy and safety of mobility-based VR program for mental health promotion needs to be performed. Conducting such a study will offer stronger evidence regarding the advantages and disadvantages of utilizing VR technology for mental health interventions in the community.

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

Yean-Hwa Lee, one of authors, declares the potential competing interests as a chief executive officer (CEO) of SY Inotech which develops virtual reality programs. Other authors declare that they have no conflicts of interest.

Author Contributions

Conceptualization: Eunsoo Moon, Myung-Jun Shin. Data curation: Hyun Ju Lim, Jeonghyun Park, Eunsoo Moon. Formal analysis: Hyun Ju Lim, Kyungwon Kim. Funding acquisition: Eunsoo Moon, Yean-Hwa Lee. Investigation: Hyebin Ko, Hyun Ju Lim, Jeonghyun Park, Eunsoo Moon. Methodology: Hyebin Ko, Hwagyu Suh, Eunsoo Moon, Du-Ri Kim, Jong-Hwan Park, Myung-Jun Shin. Project administration: Eunsoo Moon. Resources: Eunsoo Moon. Software: Hyun Ju Lim, Kyungwon Kim, Yean-Hwa Lee. Supervision: Kyungwon Kim, Hwagyu Suh, Byung Dae Lee, Young Min Lee, Jong-Hwan Park, Myung-Jun Shin. Validation: Hwagyu Suh, Byung Dae Lee, Young Min Lee. Visualization: Hyun Ju Lim, Jeonghyun Park. Writing—roriginal draft: Hyebin Ko, Hyun Ju Lim, Eunsoo Moon. Wrting—rreview & editing: Hyun Ju Lim, Jeonghyun Park, Kyungwon Kim, Hwagyu Suh, Byung Dae Lee, Young Min Lee, Eunsoo Moon, Du-Ri Kim, Jong-Hwan Park, Myung-Jun Shin, Yean-Hwa Lee.

Funding Statement

This research was financially supported by the Busan Metropolitan City, Korea, under the ‘Mobility-based flexible medical service’.

ACKNOWLEDGEMENTS

None

Figure 1.
Process of VR-assisted mental health promotion program in the bus for community population. VR, virtual reality; PHQ-9, Patient Health Questionnaire-9; GAD-7, Generalized Anxiety Disorder-7; PSS, Perceived Stress Scale; WHOQOL-BREF, World Health Organization Quality of Life Scale Abbreviated Version.
pi-2023-0286f1.tif
Figure 2.
Proportion of subjective satisfaction on virtual reality program.
pi-2023-0286f2.tif
Table 1.
Demographic and clinical characteristics of the participants (N=96)
Variables Value
Age, year 72.8 (±17.2)
Female 72 (75.0)
Education year 9.2 (±5.0)
Marital status
 Unmarried 7 (7.3)
 Married/cohabitated 43 (44.8)
 Divorced/separated 42 (43.8)
 Others 4 (4.2)
Job(+) 53 (55.2)
PHQ-9 3.4 (±4.4)
GAD-7 1.9 (±3.1)
PSS 20.8 (±7.4)
WHOQOL-BREF 79.6 (±23.2)
PHQ(+) (high-risk proportion) 14 (14.6)
GAD(+) (high-risk proportion) 2 (2.1)
Subjective depressive feeling (score 0–100) 47.1 (±20.3)
Subjective stressful feeling (score 0–100) 47.3 (±21.0)
Subjective satisfaction (score 0–100) 74.2 (±19.2)

Data were represented with mean (±SD) or number (%) according to the characteristics of variables. PHQ-9, Patient Health Questionnaire-9; GAD-7, Generalized Anxiety Disorder-7; PSS, Perceived Stress Scale; WHOQOL-BREF, World Health Organization Quality of Life Scale Abbreviated Version

Table 2.
Correlation analysis between clinical characteristics by subjective self-rating scales and subjective feelings during VR training
Subjective depressive feeling Subjective stressful feeling
PHQ-9 0.307 0.274*
GAD-7 0.299 0.248*
PSS 0.258* 0.107
WHOQOL-BREF 0.011 -0.017

Statistics was tested by Spearman correlation analysis.

* p<0.05;

p<0.01.

VR, virtual reality; PHQ-9, Patient Health Questionnaire-9; GAD-7, Generalized Anxiety Disorder-7; PSS, Perceived Stress Scale; WHOQOL-BREF, World Health Organization Quality of Life Scale Abbreviated Version

Table 3.
Changes of subjective feelings before and after experience of VR-assisted smart mental care (N=96)
Before VR After VR Z (p)
Subjective depressive feeling 47.1 (±20.3) 43.1 (±17.7) -2.092 (0.036)
Subjective stressful feeling 47.3 (±21.0) 43.5 (±17.9) -2.592 (0.010)

Statistics was tested by Wilcoxon signed-rank test between before and after VR. VR, virtual reality

Table 4.
Changes of subjective feelings before and after experience of VR-assisted smart mental care
PHQ-9(+) subjects (N=14)
PHQ-9(-) subjects (N=82)
Before VR After VR Z (p) Before VR After VR Z (p)
Subjective depressive feeling 64.29 (±17.9) 52.86 (±20.2) -2.126 (0.033) 44.15 (±19.3) 41.46 (±16.9) -1.335 (0.182)
Subjective stressful feeling 61.43 (±22.8) 51.43 (±17.0) -2.111 (0.035) 44.88 (±19.8) 42.20 (±17.8) -1.895 (0.058)

Statistics was tested by Wilcoxon signed-rank test between before and after VR. VR, virtual reality; PHQ-9, Patient Health Questionnaire-9

REFERENCES

1. Thornicroft G, Tansella M. Components of a modern mental health service: a pragmatic balance of community and hospital care: overview of systematic evidence. Br J Psychiatry 2004;185:283–290.
crossref pmid
2. Parcesepe AM, Cabassa LJ. Public stigma of mental illness in the United States: a systematic literature review. Adm Policy Ment Health 2013;40:384–399.
crossref
3. Pescosolido BA, Halpern-Manners A, Luo L, Perry B. Trends in public stigma of mental illness in the US, 1996-2018. JAMA Netw Open 2021;4:e2140202.
crossref pmid pmc
4. Keyes CL. Promoting and protecting mental health as flourishing: a complementary strategy for improving national mental health. Am Psychol 2007;62:95–108.
crossref pmid
5. Bolier L, Haverman M, Westerhof GJ, Riper H, Smit F, Bohlmeijer E. Positive psychology interventions: a meta-analysis of randomized controlled studies. BMC Public Health 2013;13:119
crossref pmid pmc
6. Delibas DH, Erdogan E. Effects of a psychoeducation program on disease burden, depression, and anxiety levels in relatives of psychotic patients in a community mental health center. Perspect Psychiatr Care 2022;58:940–945.
crossref pmid
7. Buechner H, Toparlak SM, Ostinelli EG, Shokraneh F, Nicholls-Mindlin J, Cipriani A, et al. Community interventions for anxiety and depression in adults and young people: a systematic review. Aust N Z J Psychiatry 2023;57:1223–1242.
crossref pmid pmc
8. Van Citters AD, Bartels SJ. A systematic review of the effectiveness of community-based mental health outreach services for older adults. Psychiatr Serv 2004;55:1237–1249.
crossref pmid
9. Giebel C, Shrestha N, Reilly S, White RG, Zuluaga MI, Saldarriaga G, et al. Community-based mental health and well-being interventions for older adults in low- and middle-income countries: a systematic review and meta-analysis. BMC Geriatr 2022;22:773
crossref pmid pmc
10. Mavrogiorgou P, Böhme P, Hooge V, Pfeiffer T, Juckel G. [Virtual reality in teaching of psychiatry and psychotherapy at medical school]. Nervenarzt 2022;93:728–734. German.
crossref pmid pmc
11. Salehi E, Mehrabi M, Fatehi F, Salehi A. Virtual reality therapy for social phobia: a scoping review. Stud Health Technol Inform 2020;270:713–717.

12. Elphinston RA, Vaezipour A, Fowler JA, Russell TG, Sterling M. Psychological therapy using virtual reality for treatment of driving phobia: a systematic review. Disabil Rehabil 2023;45:1582–1594.
crossref
13. Meyerbröker K, Morina N. The use of virtual reality in assessment and treatment of anxiety and related disorders. Clin Psychol Psychother 2021;28:466–476.
crossref pmid pmc
14. Jingili N, Oyelere SS, Ojwang F, Agbo FJ, Nyström MBT. Virtual reality for addressing depression and anxiety: a bibliometric analysis. Int J Environ Res Public Health 2023;20:5621
crossref pmid pmc
15. Baghaei N, Chitale V, Hlasnik A, Stemmet L, Liang HN, Porter R. Virtual reality for supporting the treatment of depression and anxiety: scoping review. JMIR Ment Health 2021;8:e29681.
crossref pmid pmc
16. Kim DR, Park JH, Lim HJ, Shin MJ, Lee YH, Moon ES. The effect of bio-signal-based virtual reality smart mental care programs on the mental health and quality of life of middle-aged and older adults. J Korea Aging Friendly Ind Assoc 2022;14:41–51.
crossref
17. Lim HJ, Kim K, Moon E, Kim DR, Park JH, Shin MJ, et al. Difference in cognitive performance in virtual reality–assisted mental health promotion program according to groups clustered based on mental health. Mood Emot 2022;20:43–51.
crossref
18. Valmaggia LR, Latif L, Kempton MJ, Rus-Calafell M. Virtual reality in the psychological treatment for mental health problems: an systematic review of recent evidence. Psychiatry Res 2016;236:189–195.
crossref
19. Conner NO, Freeman HR, Jones JA, Luczak T, Carruth D, Knight AC, et al. Virtual reality induced symptoms and effects: concerns, causes, assessment & mitigation. Virtual Worlds 2022;1:130–146.
crossref
20. Park S, Lee G. Full-immersion virtual reality: adverse effects related to static balance. Neurosci Lett 2020;733:134974
crossref pmid
21. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606–613.
crossref pmid pmc
22. An JY, Seo ER, Lim KH, Shin JH, Kim JB. Standardization of the Korean version of screening tool for depression (Patient Health Questionnaire-9, PHQ-9). J Korean Soc Biol Ther Psychiatry 2013;19:47–56.

23. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166:1092–1097.
crossref pmid
24. Lee SH, Shin C, Kim H, Jeon SW, Yoon HK, Ko YH, et al. Validation of the Korean version of the Generalized Anxiety Disorder 7 self-rating scale. Asia Pac Psychiatry 2022;14:e12421.
crossref
25. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav 1983;24:385–396.
crossref pmid
26. Hong GR, Kang HK, Oh E, Park Y, Kim H. Reliability and validity of the Korean version of the Perceived Stress Scale-10 (K-PSS-10) in older adults. Res Gerontol Nurs 2016;9:45–51.
crossref pmid
27. Skevington SM, Lotfy M, O’Connell KA, WHOQOL Group. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. A report from the WHOQOL group. Qual Life Res 2004;13:299–310.
crossref
28. Min SK, Lee CI, Kim KI, Suh SY, Kim DK. Development of Korean version of WHO Quality of Life Scale Abbreviated Version (WHOQOL-BREF). J Korean Neuropsychiatr Assoc 2000;39:571–579.

29. Manea L, Gilbody S, McMillan D. Optimal cut-off score for diagnosing depression with the Patient Health Questionnaire (PHQ-9): a meta-analysis. CMAJ 2012;184:E191–E196.
crossref pmid pmc
30. Carl E, Stein AT, Levihn-Coon A, Pogue JR, Rothbaum B, Emmelkamp P, et al. Virtual reality exposure therapy for anxiety and related disorders: a meta-analysis of randomized controlled trials. J Anxiety Disord 2019;61:27–36.
crossref pmid
31. Gerardi M, Cukor J, Difede J, Rizzo A, Rothbaum BO. Virtual reality exposure therapy for post-traumatic stress disorder and other anxiety disorders. Curr Psychiatry Rep 2010;12:298–305.
crossref pmid
32. Gorini A, Pallavicini F, Algeri D, Repetto C, Gaggioli A, Riva G. Virtual reality in the treatment of generalized anxiety disorders. Stud Health Technol Inform 2010;154:39–43.
pmid
33. Arlati S, Colombo V, Spoladore D, Greci L, Pedroli E, Serino S, et al. A social virtual reality-based application for the physical and cognitive training of the elderly at home. Sensors (Basel) 2019;19:261
crossref pmid pmc
34. Fils-Aimé JR, Grelotti DJ, Thérosmé T, Kaiser BN, Raviola G, Alcindor Y, et al. A mobile clinic approach to the delivery of community-based mental health services in rural Haiti. PLoS One 2018;13:e0199313.
crossref pmid pmc
35. Head D, Isom M. Age effects on wayfinding and route learning skills. Behav Brain Res 2010;209:49–58.
crossref pmid
36. Gorini A, Mosso JL, Mosso D, Pineda E, Ruíz NL, Ramíez M, et al. Emotional response to virtual reality exposure across different cultures: the role of the attribution process. Cyberpsychol Behav 2009;12:699–705.
crossref pmid
37. Shirazi F, Nasab NZ, Jaberi A. Comparing the effects of virtual reality and home chair-based exercises on balance, daily living activities, and loneliness among older adults with balance disorders. Res Gerontol Nurs 2023;16:33–42.
crossref pmid
38. Rose V, Stewart I, Jenkins KG, Tabbaa L, Ang CS, Matsangidou M. Bringing the outside in: the feasibility of virtual reality with people with dementia in an inpatient psychiatric care setting. Dementia (London) 2021;20:106–129.
crossref pmid