Effects of Virtual Reality-Based Cognitive Training in the Elderly with and without Mild Cognitive Impairment

Article information

Psychiatry Investig. 2021;18(7):619-627
Publication date (electronic) : 2021 July 16
doi : https://doi.org/10.30773/pi.2020.0446
1Department of Psychiatry, Inha University Hospital, Incheon, Republic of Korea
2Department of Psychiatry, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Republic of Korea
3Department of Psychiatry, Gachon University Gil Medical Center, Incheon, Republic of Korea
4Department of Computer Engineering, Gachon University, Seongnam, Republic of Korea
5Department of IT Convergence Engineering, Gachon University Graduate School, Seongnam, Republic of Korea
Correspondence: Jae Nam Bae, MD, PhD Department of Psychiatry, Inha University College of Medicine, 27 Inhangro, Jung-gu, Incheon 22332, Republic of Korea Tel: +82-32-890-3592, Fax: +82-32-890-3558, E-mail: jnbae@inha.ac.kr
Correspondence: Seong-Jin Cho, MD, PhD Department of Psychiatry, Gachon University Gil Medical Center, 21 Namdong-daero 774beon-gil, Namdong-gu, Incheon 21565, Republic of Korea Tel: +82-32-458-2841, Fax: +82-32-472-3396, E-mail: sjcho@gilhospital.com
Received 2020 December 30; Revised 2021 March 9; Accepted 2021 April 11.

Abstract

Objective

This study aimed to introduce a 4-week long fully immersive virtual reality-based cognitive training (VRCT) program that could be applied for both a cognitively normal elderly population and patients with mild cognitive impairment (MCI). In addition, we attempted to investigate the neuropsychological effects of the VRCT program in each group.

Methods

A total of 56 participants, 31 in the MCI group and 25 in the cognitively normal elderly group, underwent eight sessions of VRCT for 4 weeks. In order to evaluate the effects of the VRCT, the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet was administered before and after the program. The program’ s safety was assessed using a simulator sickness questionnaire (SSQ), and availability was assessed using the presence questionnaire.

Results

After the eighth session of the VRCT program, cognitive improvement was observed in the ability to learn new information, visuospatial constructional ability, and frontal lobe function in both groups. At the baseline evaluation, based on the SSQ, the MCI group complained of disorientation and nausea significantly more than the cognitively normal elderly group did. However, both groups showed a reduction in discomfort as the VRCT program progressed.

Conclusion

We conclude that our VRCT program helps improve cognition in both the MCI group and cognitively normal elderly group. Therefore, the VRCT is expected to help improve cognitive function in elderly populations with and without MCI.

INTRODUCTION

Dementia is one of the most common major diseases in the elderly. It is a progressive neurocognitive disorder that gradually progresses and causes cognitive and functional decline [1,2]. Mild cognitive impairment (MCI) is often a proximal risk factor leading to dementia. The risk of conversion from MCI to dementia is estimated to be approximately 5–10% per year. The risk increases with the number of impaired cognitive domains and symptom severity [3]. However, the variation in presentation and slow progression of the disease make it difficult to define clear boundaries between MCI and dementia.

Early treatment of MCI has been suggested to slow the progression of dementia [4]. Acetylcholinesterase inhibitors are mainly used in the pharmacological treatment for dementia [5], but they are limited in their ability to prevent disease progression. Cognitive therapy is mainly used as a non-pharmaceutical intervention to maintain the independence of patients [6].

Virtual reality-based cognitive therapy (VRCT) and computerized cognitive training (CCT) have been introduced as new methods of training. CCT is known to benefit global cognition, selective cognitive domains, and psychosocial functioning [7]. VRCT provides an immersive environment that allows users to simultaneously experience various sensations, such as sight, hearing, and touch, to offer similar experiences to those faced in real life. As a result, attention and spatial memory training becomes possible, motivation-inducing effect increases, and concomitantly reduces the time and effort required in treatment [8-10].

It is difficult to generalize the effects of VRCT as the nature of the cognitive training varies according to the method of VR application, frequency of sessions, and the immersion of the provided environment. In a systematic review of technology-based cognitive interventions in patients with MCI, VR was used in only two of 26 studies [11]. A recent meta-analysis study showed that VRCT was used in only six out of 11 studies, and only one study out of those six applied a fully immersive technique [12]. VRCT in the healthy elderly population is also known to have a significant effect on cognitive function, primarily seen as improvements in attention, executive function, and memory [13]. However, it is difficult to determine the definitive effects of VRCT as the type of technology applied and training period have not been consistent.

Therefore, this study aimed to introduce a fully immersive eight-session of VRCT program that can be applied for both a cognitively normal elderly population and patients with MCI. Moreover, the effects of the program were assessed using the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet (CERAD-K) neuropsychological evaluation.

METHODS

Participants

From October 1, 2018, to June 30, 2019, we recruited patients who visited three general hospitals in South Korea, Gil Hospital, Inha University Hospital, and Samsung Medical Center, complaining of memory loss. Patients who met the criteria for MCI were selected based on the Petersen criteria [14]: 1) memory problems, 2) objective memory disorder, 3) absence of other cognitive disorders or repercussions on daily life, 4) normal general cognitive function, and 5) absence of dementia. Participants were considered cognitively normal when they performed within 1.5 standard deviations of age-adjusted norms on CERAD-K. Participants who were unable to communicate or had severe medical disorders, organic mental disorders, intellectual disorders, autistic spectrum disorders, acute psychotic symptoms, or major neurocognitive disorders were excluded from the study.

All participants provided informed consent. This study protocol was approved by the Institutional Review Board at each study site (INHA 2018-06-021, GAIRB2018-052, 2018-07-045).

Virtual reality-based cognitive therapy (VRCT)

All participants were trained using a newly developed fully immersive 3D VRCT program [15]. The program consists of purchasing goods at a supermarket and has been reconstructed as contents suitable for a fully immersive 3D VR environment. Fully immersive technology is a fundamentally different user experience compared to partial immersion, while partial immersion supports the feeling of “looking at” the virtual invironment, while full immersion supports the feeling of “being in” that environment. The degree of immersion in the virtual environment is measured according to how much the user’s field of vew (fov) can be covered. If the display provides an alwayson field of view, it can be defined as a fully immersive display [16]. It was developed using the C# scripting language.

The program was conducted over a total of eight sessions over 4 weeks with individual sessions lasting for 50 to 60 minutes. Twenty-five minutes of the session were spent on receiving feedback and strategy planning based on the previous session. The VRCT was then supplied for 30–35 minutes. Training was conducted using a Samsung Odyssey head mounted display and motion controller in each hand; these devices are commercially available. Except for one participant in a wheelchair, VRCT for all others was performed standing up. A psychologist assisted the participants in their performance.

In the first session, program introduction, device use training, goal setting, memory strategy training, and VRCT were conducted. The second session consisted of conversations about daily life since the last session, memory strategy education, and VRCT. The third to eighth sessions were conducted in the same way as the second session, and the eighth session was the last session and concluded by reviewing the participant’s progress based on the acquisition of memory strategies and performance results. At each session, a review of daily life since the last session and retraining of the memory strategy were conducted, except for in the first session.

The VRCT program was designed to train memory, attention, and executive function. It was a 3D VR program where participants were required to purchase necessary items in a supermarket. Participants were given a list of items to buy for 10 seconds at the entrance of the supermarket, and then they visited seven sections by pushing a button. In each session, items were evenly and randomly selected from seven categories, such as juice, dairy foods, meat, fish, alcohol, processed food, and Korean side dishes. There were four levels of difficulty from level 1 (four items to buy) to level 4 (seven items to buy); participants chose the difficulty level according to their performance in each session. During training, participants could use a hint button, which led to the list of items to buy being displayed for 10 seconds. Examples of the VRCT program are presented in Figure 1.

Figure 1.

Representative images of virtual reality cognitive training (VRCT) program. A: Participants choose the level of program from level 1 (four items) to level 4 (seven items) at the entrance of the supermarket. B: Participants are given a list of items to buy from categories such as juice (C), dairy food (D), meat (E), fish (F), alcohol (G), processed food (H), and Korean side dishes (I). J: After buying all items they automatically move to the counter. K: Participants can use the hint button and the list of items to buy is displayed again for 10 seconds anytime during training. L: Example of a participant.

Measures

Clinical scales

A Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet (CERAD-K)

The CERAD-K [17] was used to evaluate each cognitive function in participants. The CERAD-K comprehensively evaluates cognitive domains related to dementia and can be used as an indicator of performance and overall cognitive function. The CERAD-K was performed before and after the VRCT program.

Korean version of Quality of Life-Alzheimer’s Disease (KQOL-AD)

KQOL-AD [18] was used to evaluate the satisfaction of participants with their lives. The scale consists of 13 questions and includes physical health, mood, memory, interpersonal relationships, and the ability to perform activities of daily living. The total score ranges from 13 to 52, with higher scores indicating higher quality of life. The KQOL-AD was also performed before and after the VRCT program.

Geriatric depression scale (GDS)

The GDS [19] was used to evaluate the degree of depression in old age. Its reliability and validity have already been confirmed. The GDS consists of 30 “yes or no” questions. Depression is suspected with a score of 18 or more out of a total of 30 points. The GDS score was also assessed before and after the VRCT program.

Safety and availability evaluation scales

Simulator sickness questionnaire

To evaluate the safety of VR devices, a simulator sickness questionnaire (SSQ) [20] was performed after the VRCT in each session. SSQ is used as a representative questionnaire to measure VR sickness as research using VR started to increase in the 2010s. It consists of 16 items across three categories: disorientation, oculomotor, and nausea, and each item is scored from 0 to 3 points. It calculates the sum of different weights applied to each item, with higher scores indicating higher discomfort [21].

Presence questionnaire

Presence is a term that refers to how realistic and immersive the experience is in technologies such as VR. In this study, two scales were modified and used by referring to the presence questionnaire introduced in 1998 [22] and the presence inventory of Lombard et al. [23]. The presence questionnaire consisted of nine items, scored from 1 to 7 for each item with higher scores reflecting a higher sense of presence. Another questionnaire consisted of 18 questions to measure the presence of VR content users and evaluated how immersive and present the patient felt during the VR experience. Each item was scored from 0 to 10 points, with higher scores indicating a higher sense of presence [15].

Statistical analyses

A chi-square test was performed to verify the homogeneity of the distribution of variables between groups. An independent two-sample t-test was performed to verify differences in the neuropsychological evaluation z-score between groups. In order to verify the effects of the program, a repeated measures analysis of variance (ANOVA) was performed on the scores of the CERAD-K for each domain, GDS, and KQOL-AD for the cognitively normal and MCI groups. SPSS 20.0 (for Windows, IBM Corp., Armonk, NY, USA) was used for all statistical analyses, and a p-value of 0.05 or less was considered statistically significant.

RESULTS

Demographic characteristics

Fifty-six out of 64 participants who wished to participate in the study were included in the analysis. Of the eight participants excluded from the analysis, six participants did not meet the study schedule criteria, one subject withdrew due to having difficulties in training while standing, and one withdrew because of experiencing dizziness when using the VR device. The MCI group included 31 participants (55.4%), and the normal group consisted of 25 participants (44.6%). Of these 56 participants, 47 participated in the study (Figure 2). There were no statistical differences in age and sex between the group completing the study and the dropout group. And also, there were no differences in terms of age, sex, educational level, employment status, or subjective quality of life between the MCI group and the normal group. The GDS score was higher in the MCI group than that in the normal group; but the score was not indicative of depression (Table 1).

Figure 2.

Flow diagram of participants’ recruitment. HMD: head-mounted display, MCI: mild cognitive impairment.

Participant demographics

Baseline cognitive performance

There was a significant functional difference between the two groups for all items except for digit span and trail making test part A (TMT-A) (Table 2).

Comparisons of the z-score for the baseline Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet between the MCI and cognitively normal groups

Pre/post-VRCT z-score effect verification

Repeated measures ANOVA was performed to verify groupspecific changes following program implementation. Among the neuropsychological test items, word list memory, constructional recall, Stroop-C, and Stroop-CW showed statistically significant changes in both the MCI group and normal group, and the effects of the VRCT program could be verified. Moreover, performance in the Korean-Boston naming test was significantly improved in the normal group. In the MCI group, the effects of the VRCT program on word list recall, word list recognition, and TMT-A performance were observed. KQOL-AD and GDS scores were not affected by the VRCT program (Table 3).

Z-score of the CERAD-K, KQOL-AD, and GDS at baseline and post-training in the MCI and cognitively normal groups

Assessment of safety and presence

Three participants of the MCI group (n=31) and one of the normal group (n=25) were excluded from the study due to dizziness. One participant of the MCI group dropped out due to pain at a past surgical site caused by wearing the head-mounted display. Another participant withdrew due to ocular discomfort.

Repeated measures ANOVA was performed to verify the group-specific changes in safety and presence after the VRCT program. At the baseline SSQ evaluation, the MCI group complained of disorientation and nausea significantly more than the normal group did (Table 4). However, both groups showed an improvement in discomfort in all items of the SSQ as the VRCT program progressed. A two-sample t-test was conducted using the SSQ that was completed after the end of the eighth session. There were no statistically significant differences in any of the three SSQ categories or the total score, suggesting that the VRCT program in this study was applicable not only to the normal group but also to the MCI group (Table 5).

Comparisons of the SSQ and presence questionnaire responses between the MCI and cognitively normal groups

Post-training comparisons of the SSQ between the MCI and cognitively normal groups

DISCUSSION

After the eighth session of the VRCT program, improvements in cognitive function over multiple domains was observed in both the MCI and normal groups. In both groups, improvements in the ability to learn new information, visuospatial constructional ability, and frontal lobe function were observed. Similar results have been observed in previous studies [24,25]. It is thought that these findings are observed as the learning environment using VR is mainly based on visual and auditory stimulation. Cognitive training in this study was also designed to train memory, attention, and executive functions and provided a visual scene of grocery shopping in a supermarket. In particular, detailed manipulation is required to observe and pick up items written on the shopping list. This training improved cognitive flexibility and selective attention, which also led to improvement in Stroop test performance. The improvement in verbal memory, visuospatial and constructional ability, attention, and executive function in the MCI group after the VRCT program is consistent with the results of a systematic review that confirmed an effect size above moderate in most areas [7]. Functional improvement in the normal group was mostly observed in areas similar to those in the MCI group; however, a statistically significant improvement was not observed in the areas of delayed recall and recognition. It is possible that statistical significance was not reached as the normal group already had a higher functional level than that of the MCI group; however, statistical significance was observed in other similar cognitive domains. Therefore, if a large number of participants are included and long-term studies are conducted, delayed recall and recognition change can also be additionally confirmed.

Disorientation and nausea were observed as possible side effects of the VRCT program. In a meta-analysis of SSQ scores in the VR environment for all ages, the disorientation subscale had an average score of 22.06, oculomotor had an average score of 16.12, and nausea had an average score of 15.76 [26]. When interpreting total SSQ scores, according to Kennedy et al. [27], scores between 10 and 15 indicate significant symptoms, scores between 15 and 20 are concerning, and scores over 20 indicate a problem. Based on this result, the VRCT used in our study is thought to cause a significant concern regarding the level of discomfort. Especially regarding the disorientation subscale in the SSQ, discomfort occurred at a problematic level. It is thought that the standing position used in this program increased the experienced discomfort. However, the cut-off score suggested in the previous study [27] was based on soldiers and has limitations in its application to the general public, especially the elderly, and was evaluated when using a flight simulator (a ground-training device that reproduces the conditions experienced on the flight deck of an aircraft trainer). In general, SSQ scores are higher in VR environments than in flight simulator environments. Therefore, the actual risk in this study was considered to be lower than that suggested by the measured scores. Discomfort at the beginning of the program seemed to increase the likelihood of post-training rejection as most dropouts complained of discomfort within the initial two sessions. However, most participants who did not drop out demonstrated increased compliance with repeated program performance. There is a possibility that the characteristics of the elderly who were highly reluctant to participate in the unfamiliar environment were reflected in the high dropouts that occurred at the beginning of the session. In other studies in which an age-related analysis related to simulator sickness was conducted, simulator sickness of the middle-aged and the elderly was similar, but there were relatively many complaints of discomfort with dropout and VR equipment among the elderly [28,29]. Therefore, to overcome such limitations, it is necessary to provide sufficient explanation and encouragement.

In the presence assessment to confirm the effectiveness of the VRCT program, no difference was observed between the MCI and the normal groups. Presence is a subjective indicator of feeling ‘being there’ in VR. Although VR is not a real event, people think of it as real reality. Presence is affected by perception, emotion, and cognition [30]. In patients with MCI, cognitive decline and perceptual impairment are common, suggesting that the sense of presence felt during VR experience may be different from that of cognitively normal people. However, no difference in presence was observed between the MCI group and the normal group in this study. This means that the VRCT program used in this study is sufficiently applicable to the MCI group with reduced cognitive ability.

The VRCT program used in this study provided a fully immersive virtual environment by using a head-mounted 3D display, and a motion controller was used in both hands so that the participants could feel the sensation of picking up objects. Moreover, an interactive design that allowed participants to obtain hints during the program was applied so that the training could be applied to daily life as quickly as possible. Moreover, in the VRCT, the entire process of training was recorded and accessed immediately. This immediate response system allowed the difficulty level to be adjusted according to the participant’s ability, which stimulated the participant’s motivation more actively [31]. In the VRCT used in this study, the difficulty of the program was adjusted according to the ability of the participants to perform at each stage, and hints were made available. These points are thought to have motivated the participants to execute the program despite being in an unfamiliar environment.

Similar to that of other VR studies, this study had a few limitations in predicting the statistical validity of the number of samples as it was conducted using a small sample size. In addition, the long-term effects of this program cannot be generalized as cognitive training was performed only eight times, which is a relatively short program. Short-term cognitive training programs for patients with MCI differ depending on the cognitive domain trained but most have proven effective [32]. In this study, only short-term effects immediately after cognitive training were assessed. Therefore, follow-up research is necessary. Furthermore, the lack of a control group makes it difficult to isolate the VRCT program-related effects. However, functional improvements in various areas were observed in the normal group after training, suggesting the possible expansion of the application of VRCT.

In this study, contrary to initial plans to increase accessibility to the program operation, practice was required for the elderly to become familiar with the program and devices, and researchers were required to assist with program operation and device use in each trial. In a study comparing the effects of VR-based and therapist-led memory training environments on subjective memory ability, the satisfaction with the memory performance in the therapist-led memory training group was much higher [24]. The authors interpreted this finding as the emotional support provided by the therapist provided reassurance during cognitive training. However, the vivid experience of fully immersive VRCT can be easily applied to a real situation, reducing time and cost compared to conventional cognitive-behavioral therapy. In addition, since the stimulus level can be adjusted according to the individual’s response, there is an advantage in that a customized treatment suitable for the individual’s cognitive level is possible. Therefore, an approach combining an easily operatable device and emotional support from therapists is required to develop a more robust VRCT program.

Although there were several limitations, the VRCT program used in this study improved function in both patients with MCI and cognitively normal elderly individuals. A meta-analysis study on the effects of VR on MCI and dementia [12] found that it was most effective target on individuals with MCI. Based on this review and our study, it seems that VRCT can improve cognitive abilities in various domains, including working memory, in normal elderly individuals and in those with MCI. Therefore, it is expected that a cognitive training program using fully immersive VR will help to improve cognitive function in a large proportion of the elderly population in the future.

Acknowledgements

This work was supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korean government (MSIT) (No. 2017-0-00180, Development of complex biosignal response information-based intelligent VR (Virtual Reality) life care technology).

This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the Information Technology Research Center (ITRC) support program (IITP-2020-2017-0-01630) supervised by the Institute for Information & Communications Technology Promotion (IITP).

Notes

The authors have no potential conflicts of interest to disclose.

Authors’ contribution

Conceptualization: all authors. Data curation: all authors. Formal analysis: Seri Maeng, Jae Nam Bae, Seong-Jin Cho. Funding acquisition: SeokHee Oh, Jae Nam Bae, Seong-Jin Cho. Investigation: Seri Maeng, Jin Pyo Hong, Won-Hyoung Kim, Hyeyoung Kim, Seo-Eun Cho, Jae Myeong Kang, Kyoung-Sae Na, Jae Nam Bae, Seong-Jin Cho. Methodology: Seri Maeng, Jin Pyo Hong, Won-Hyoung Kim, Hyeyoung Kim, Seo-Eun Cho, Jae Myeong Kang, Kyoung-Sae Na, Jae Nam Bae, Seong-Jin Cho. Project administration: Seok-Hee Oh, Jae Nam Bae, Seong-Jin Cho. Resources: all authors. Software: Seok-Hee Oh, Jung Woon Park, Jae Nam Bae, Seong-Jin Cho. Supervision: Jae Nam Bae, Seong-Jin Cho. Validation: Seri Maeng, Jin Pyo Hong, Won-Hyoung Kim, Hyeyoung Kim, Seo-Eun Cho, Jae Myeong Kang, Kyoung-Sae Na. Visualization: Seri Maeng, Jae Myeong Kang. Writing—original draft: Seri Maeng, Jae Nam Bae. Writing—review & editing: all authors.

References

1. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement 2013;9:63–75.e2.
2. Wimo A, Jönsson L, Bond J, Prince M, Winblad B. The worldwide economic impact of dementia 2010. Alzheimers Dement 2013;9:1–11.e3.
3. Knopman DS, Beiser A, Machulda MM, Fields J, Roberts RO, Pankratz VS, et al. Spectrum of cognition short of dementia: Framingham Heart Study and Mayo Clinic Study of Aging. Neurology 2015;85:1712–1721.
4. Petersen RC. Mild cognitive impairment: transition between aging and Alzheimer’s disease. Neurologia 2000;15:93–101.
5. Muayqil T, Camicioli R. Systematic review and meta-analysis of combination therapy with cholinesterase inhibitors and memantine in Alzheimer’s disease and other dementias. Dement Geriatr Cogn Dis Extra 2012;2:546–572.
6. Yamaguchi H, Maki Y, Yamagami T. Overview of non‐pharmacological intervention for dementia and principles of brain‐activating rehabilitation. Psychogeriatrics 2010;10:206–213.
7. Hill NT, Mowszowski L, Naismith SL, Chadwick VL, Valenzuela M, Lampit A. Computerized cognitive training in older adults with mild cognitive impairment or dementia: a systematic review and meta-analysis. Am J Psychiatry 2017;174:329–340.
8. Mayeux R, Sano M. Treatment of Alzheimer’s disease. New Engl J Med 1999;341:1670–1679.
9. García-Betances RI, Cabrera-Umpiérrez MF, Arredondo MT. Computerized neurocognitive interventions in the context of the brain training controversy. Rev Neurosci 2017;29:55–69.
10. Zygouris S, Ntovas K, Giakoumis D, Votis K, Doumpoulakis S, Segkouli S, et al. A preliminary study on the feasibility of using a virtual reality cognitive training application for remote detection of mild cognitive impairment. J Alzheimers Dis 2017;56:619–627.
11. Ge S, Zhu Z, Wu B, McConnell ES. Technology-based cognitive training and rehabilitation interventions for individuals with mild cognitive impairment: a systematic review. BMC Geriatr 2018;18:213.
12. Kim O, Pang Y, Kim J. The effectiveness of virtual reality for people with mild cognitive impairment or dementia: a meta-analysis. BMC Psychiatry 2019;19:219.
13. Basak C, Qin S. Virtual Cognitive Training in Healthy Aging and Mild Cognitive Impairment. In : Richard Pak, Anne Mclaughlin, eds. Technology and Health New York: Elsevier; 2018. p. 215–235.
14. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med 2004;256:183–194.
15. Park JW, Oh SH. A study on the Development of VR Contents for Improvement of MCI (Mild Cognitive Impairment). J Next Generat Converg Inform Serv Technol 2018;7:149–162.
16. Kjeldskov J. Interaction: Full and Partial Immersive Virtual Reality Displays. IRIS 24: Proceedings of the 24th Information Systems Research Seminar in Scandinavia Bergen: University of Bergen; 2001. p. 587–600.
17. Lee JH, Lee KU, Lee DY, Kim KW, Jhoo JH, Kim JH, et al. Development of the Korean Version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet (CERAD-K) clinical and neuropsychological assessment batteries. J Gerontol B Psychol Sci Soc Sci 2002;57:47–53.
18. Kim HJ, Moon SY, Kim SY, Han SH. Assessment of the quality of life in patients with Alzheimer’s disease. J Korean Neurol Assoc 2008;26:308–313.
19. Jung IK, Kwak DI, Joe SH, Lee HS. A study of standardization of Korean form of Geriatric Depression Scale (KGDS). J Korean Geriatr Psychiatr 1997;1:61–72.
20. Kennedy RS, Lane NE, Berbaum KS, Lilienthal MG. Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int J Aviat Psychol 1993;3:203–220.
21. Oh KS, Oh SH. A study on user-experience evaluation of VR contents using questionnaire. J Digital Art Engineering Multimedia 2018;5:135–146.
22. Witmer BG, Singer MJ. Measuring presence in virtual environments: A presence questionnaire. Presence 1998;7:225–240.
23. Lombard M, Ditton TB, Weinstein L. Measuring presence: the temple presence inventory. In : In Proceedings of the 12th annual international workshop on presence; 2019. 1–15.
24. Man DW, Chung JC, Lee GY. Evaluation of a virtual reality‐based memory training programme for Hong Kong Chinese older adults with questionable dementia: a pilot study. Int J Geriatr Psychiatry 2012;27:513–520.
25. Zając-Lamparska L, Wiłkość-Dębczyńska M, Wojciechowski A, Podhorecka M, Polak-Szabela A, Warchoł Ł, et al. Effects of virtual realitybased cognitive training in older adults living without and with mild dementia: a pretest–posttest design pilot study. BMC Res Notes 2019;12:776.
26. Saredakis D, Szpak A, Birckhead B, Keage HA, Rizzo A, Loetscher T. Factors associated with virtual reality sickness in head-mounted displays: a systematic review and meta-analysis. Front Hum Neurosci 2020;14:96.
27. Kennedy RS, Drexler JM, Compton DE, Stanney KM, Lanham DS, Harm DL. Configural Scoring of Simulator Sickness, Cybersickness and Space Adaptation Syndrome: Similarities and Differences. In : Hettinger LJ, Haas MW, eds. Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues Boca Raton: CRC press; 2003. p. 247–278.
28. Park GD, Allen RW, Fiorentino D, Rosenthal TJ, Cook ML. Simulator sickness scores according to symptom susceptibility, age, and gender for an older driver assessment study. Proceedings of the human factors and ergonomics society annual meeting 2006;50:2702–2706.
29. Liu L, Watson B, Miyazaki M. VR for the elderly: quantitative and qualitative differences in performance with a driving simulator. Cyberpsychol Behav 1999;2:567–576.
30. Riva G, Davide F, IJsselsteijn WA. Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments Amsterdam: IOS Press; 2003.
31. Grealy MA, Johnson DA, Rushton SK. Improving cognitive function after brain injury: the use of exercise and virtual reality. Arch Phys Med Rehabil 1999;80:661–667.
32. Li H, Li J, Li N, Li B, Wang P, Zhou T. Cognitive intervention for persons with mild cognitive impairment: a meta-analysis. Ageing Res Rev 2011;10:285–296.

Article information Continued

Figure 1.

Representative images of virtual reality cognitive training (VRCT) program. A: Participants choose the level of program from level 1 (four items) to level 4 (seven items) at the entrance of the supermarket. B: Participants are given a list of items to buy from categories such as juice (C), dairy food (D), meat (E), fish (F), alcohol (G), processed food (H), and Korean side dishes (I). J: After buying all items they automatically move to the counter. K: Participants can use the hint button and the list of items to buy is displayed again for 10 seconds anytime during training. L: Example of a participant.

Figure 2.

Flow diagram of participants’ recruitment. HMD: head-mounted display, MCI: mild cognitive impairment.

Table 1.

Participant demographics

Variables MCI group (N=31) Normal group (N=25) p-value
Age (year) Mean±SD 73.2±7.3 71.6±4.4 0.355
Male sex N (%) 8 (25.8) 3 (12.0) 0.196
Married status N (%) 17 (54.8) 14 (56.0) 0.704
Education (years) Mean±SD 9.5±4.7 8.9±3.4 0.619
Employed N (%) 6 (19.4) 6 (24.0) 0.674
KQOL-AD Mean±SD 32.0±8.2 34.6±4.2 0.168
GDS Mean±SD 12.6±6.8 7.7±4.6 0.003*
*

p<0.01.

MCI: mild cognitive impairment, KQOL-AD: Korean version of Quality of Life-Alzheimer’s Disease, GDS: Geriatric Depression Scale

Table 2.

Comparisons of the z-score for the baseline Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet between the MCI and cognitively normal groups

Variables (Z-score) Mean±SD
p-value
MCI group (N=31) Normal group (N=25)
Digit span (F) 0.5±1.2 0.3±1.1 0.494
Digit span (B) -0.1±1.1 0.1±1.1 0.555
Verbal fluency -0.7±1.0 0.2±1.1 0.002
Korean-Boston naming -0.1±1.1 0.7±0.7 0.003
Word list memory -0.7±1.0 0.7±0.8 0.000
Word list recall -1.3±1.1 0.5±0.9 0.000
Word list recognition -1.5±1.7 0.3±0.5 0.000
Constructional recall -0.9±1.1 0.3±0.8 0.000
Constructional praxis -0.5±1.1 0.1±0.9 0.033*
TMT-A -0.1±2.2 0.5±0.8 0.235
TMT-B -1.5±1.3 -0.6±1.4 0.017*
Stroop-W -0.6±1.1 0.5±1.2 0.000
Stroop-C -0.9±1.1 0.5±1.2 0.000
Stroop-CW -1.3±1.2 -0.4±1.0 0.005
*

p<0.05,

p<0.01.

Digit span (F): digit span forward, digit span (B): digit span backward, TMT-A: trail taking test part A, TMT-B: trail making test part B, Stroop-W: Stroop test (word), Stroop test (color): Stroop-CW: Stroop test (color-word), MCI: mild cognitive impairment

Table 3.

Z-score of the CERAD-K, KQOL-AD, and GDS at baseline and post-training in the MCI and cognitively normal groups

MCI group (Z score)
Normal group (Z score)
Baseline Post-training F (p-value) Baseline Post-training F (p-value)
CERAD-K
Digit span (F) 0.5±1.2 0.7±1.1 4.078 (0.056) 0.2±1.1 0.3±1.0 0.000 (0.993)
Digit span (B) -0.1±1.1 -0.2±1.2 0.003 (0.960) 0.1±1.1 -0.1±0.9 1.062 (0.314)
Verbal fluency -0.7±1.0 -0.5±0.8 0.776 (0.388) 0.2±1.1 0.2±0.8 0.123 (0.729)
Korean-Boston naming -0.1±1.1 0.2±1.3 2.852 (0.106) 0.7±0.7 1.0±0.5 11.000 (0.003)
Word list memory -0.7±1.0 0.2±1.3 24.112 (0.000) 0.7±0.8 1.5±0.9 22.856 (0.000)
Word list recall -1.3±1.1 -1.0±1.2 12.939 (0.002) 0.5±0.9 0.8±0.8 2.569 (0.124)
Word list recognition -1.5±1.7 -1.0±1.8 7.511 (0.012*) 0.3±0.5 0.4±0.5 0.959 (0.339)
Constructional recall -0.9±1.1 -0.5±1.2 7.969 (0.010*) 0.3±0.8 0.7±0.9 5.146 (0.033*)
Constructional praxis -0.5±1.1 -0.6±1.9 0.093 (0.764) 0.1±0.9 0.3±1.0 1.113 (0.303)
TMT-A -0.1±2.2 0.2±2.4 5.720 (0.026*) 0.5±0.8 0.5±1.2 0.040 (0.844)
TMT-B -1.5±1.3 -1.6±1.2 0.462 (0.504) -0.6±1.4 -0.5±1.4 0.368 (0.551)
Stroop-W -0.6±1.1 -0.6±1.4 1.714 (0.205) 0.5±0.9 0.7±0.8 1.319 (0.263)
Stroop-C -0.9±1.1 -0.7±1.2 6.891 (0.016*) 0.5±1.2 0.7±1.0 5.770 (0.025*)
Stroop-CW -1.3±1.2 -1.2±1.2 7.383 (0.013*) -0.4±1.0 -0.1±0.9 9.711 (0.005)
KQOL-AD 32.0±8.2 32.4±7.5 0.874 (0.360) 34.6±4.2 34.0±4.7 0.104 (0.750)
GDS 12.6±6.8 11.8±7.7 0.770 (0.389) 7.7±4.6 8.1±4.7 0.227 (0.639)
*

p<0.05,

p<0.01.

Digit span (F): digit span forward, digit span (B): digit span backward, TMT-A: trail making test part A, TMT-B: trail making test part B, Stroop-W: Stroop test (Word), Stroop-C: Stroop test (Color), Stroop-CW: Stroop test (Color-Word), KQOL-AD: Korean version of Quality of Life-Alzheimer’s Disease, GDS: Geriatric Depression Scale, CERAD-K: Korean version of the Consortium to Establish a Registry for alzheimer’s disease Assessment Packet, MCI: mild cognitive impairment

Table 4.

Comparisons of the SSQ and presence questionnaire responses between the MCI and cognitively normal groups

MCI group
Normal group
F (p-value)
Baseline Post-training Baseline Post-training
VR-sickness
 SSQ-N 29.85±32.56 14.46±21.17 14.50±17.87 10.30±17.61 5.398 (0.024*)
 SSQ-D 83.07±75.51 32.33±46.91 46.77±52.22 26.16±37.76 4.279 (0.043*)
 SSQ-O 26.16±33.66 11.24±19.26 17.58±21.29 10.91±17.10 0.803 (0.374)
 SSQ-TS 46.93±45.19 19.90±29.45 26.92±30.32 16.45±24.42 3.383 (0.071)
Presence
 Presence Q 43.50±10.00 43.29±10.00 43.13±9.15 47.65±9.71 0.724 (0.399)
 Presence S 115.79±27.68 107.45±20.98 115.78±21.59 118.78±21.69 0.909 (0.346)
*

p<0.05.

SSQ-N: simulator sickness questionnaire-nausea, SSQ-D: simulator sickness questionnaire-disorientation, SSQ-O: simulator sickness questionnaire-oculomotor, SSQ-TS: simulator sickness questionnaire-total score [SSQ-TS=(SSQ-N+SSQ-D+SSQ-O)×3.74], Presence Q: questionnaire for measuring presence of virtual reality contents, Presence S: questionnaire consisting of 10 points scale for measuring the presence of virtual reality content users, MCI: mild cognitive impairment

Table 5.

Post-training comparisons of the SSQ between the MCI and cognitively normal groups

VR-sickness Mean ± SD
p-value
MCI group Normal group
SSQ-N 14.46±21.17 10.30±17.61 0.435
SSQ-D 32.33±46.91 26.16±37.76 0.597
SSQ-O 11.24±19.26 10.91±17.10 0.946
SSQ-TS 19.90±29.45 16.45±24.42 0.641

MCI: mild cognitive impairment, SSQ-N: simulator sickness questionnaire-nausea, SSQ-D: simulator sickness questionnairedisorientation, SSQ-O: simulator sickness questionnaire-oculomotor, SSQ-TS: simulator sickness questionnaire-total score