Validation of the Korean Version of the Coronavirus Reassurance-Seeking Behaviors Scale During the COVID-19 Pandemic
Article information
Abstract
Objective
In this study, we aimed to develop a Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale (CRBS) and to explore its reliability and validity among the general population in South Korea.
Methods
Using an online survey conducted during November 9–15, 2021, we collected the demographic data of 400 individuals and their responses to rating scales such as the CRBS, the Stress and Anxiety to Viral Epidemics-6 items (SAVE-6), the Generalized Anxiety Disorder-7 items (GAD-7), and the Patient Health Questionnaire-9 items (PHQ-9). We conducted factor analyses and utilized item response theory to confirm the validity and reliability of the Korean version of the CRBS.
Results
Factor analyses revealed that the single factor model of the Korean version of the CRBS showed a good fit with the CRBS (χ2=5.475, df=5, p value=0.361, χ2/df=1.095, CFI=0.999, TLI=0.998, RMSEA=0.015). Multigroup CFA results indicated that the CRBS measures reassurance-seeking behaviors consistently across variables of sex, depression, general anxiety, and viral anxiety. The CRBS also exhibited good convergent validity with the SAVE-6 (r=0.431, p<0.001), GAD-7 (r=0.574, p<0.001), and PHQ-9 (r=0.575, p<0.001).
Conclusion
The CRBS is a reliable and valid rating scale that measures reassurance-seeking behavior in relation to viral epidemics.
INTRODUCTION
Since the first case of coronavirus disease (COVID-19) [1], the virus has rapidly spread all over the world. A total of 481,756,671 confirmed cases and 6,127,981 deaths have been reported globally to date [2]. In South Korea, 12,774,956 cases have been confirmed and 15,855 deaths [3] have been noted as of March 30, 2022. In this pandemic, people suffer psychological problems such as depression, anxiety, insomnia, or post-traumatic stress disorder, adding to the stress (29.6%), anxiety (31.9%), and depression (33.7%) prevalent in the general population [4]. The number of confirmed cases has rapidly increased and the South Korean government has recently announced a policy acknowledging the continued association of human life with the coronavirus [5].
In this pandemic, people tend to be anxious about being infected and incurring physical illness or mental dysfunction. Based on the Illness Anxiety Disorder model [6], fear can provoke reassurance-seeking behavior, which may relieve fear or anxiety transiently. However, repetitive reassurance-seeking behavior can lead, in turn, to increased anxiety, enhancing a preoccupation with the causes of the fear. It was reported that 24% of dental students were assessed as having a severe degree of reassurance-seeking behavior [7]. Reassurance-seeking behaviors have been associated with viral anxiety, health anxiety, and death anxiety [8] among the Indian population and health worry or fear of illness and death [9] among the general population in Iraq. In this pandemic, an individual’s viral anxiety can provoke reassurance-seeking behaviors such as washing hands, checking body temperature, monitoring physical symptoms, or collecting information on COVID-19 by searching the internet. These behaviors can reduce viral anxiety transiently, but repetitive behaviors can generate preoccupation with COVID-19. Therefore, we need to assess our degree of reassurance-seeking behaviors and develop guidelines for public mental health policy.
Previously, several rating scales measuring reassurance-seeking behaviors such as the Reassurance-Seeking Scale [10,11], the Reassurance-Seeking Questionnaire [12], and the Threat-related Reassurance-Seeking Scale [13] were developed. However, these scales were not specific to the viral epidemic. To understand people’s motives for seeking reassurance in order to reduce anxiety or fear related to COVID-19 infection, a proper assessment scale which is able to measure viral epidemic-specific, reassurance-seeking behaviors is required. The Coronavirus Reassurance-Seeking Behaviors Scale (CRBS) [14] was developed to measure how frequently individuals engaged in reassuranceseeking activities, specifically in relation to the coronavirus. In the same study, the scale was shown to have a good internal consistency (Cronbach’s α=0.90), although the structural validity of the scale was not revealed. In other research, the CRBS was studied and reported to correlate with neuroticism or hypochondriasis [15]. In this study, we aimed to develop a Korean version of the CRBS and to explore its validity and reliability among the general population in South Korea.
METHODS
Participants and procedure
We conducted an anonymous online survey on November 9–15, 2021, among the general population of South Korea. All 400 individuals participated in this study via EMBRAIN (www.embrain.com), a professional survey company. We collected participants’ information on age, sex, living region, and marital status, but we did not collect any identifiable personal information. We also gathered responses to questions on COVID-19 such as: “Did you experience being quarantined due to infection with COVID-19?,” “Did you experience being infected with COVID-19?,” “Did you get vaccinated?”
Participants’ past psychiatric history was investigated with the following question: “Have you ever experienced depression, anxiety, or insomnia and had treatment for it?” Current psychiatric distress was probed with the question: “Do you think you are depressed or anxious now and do you need help with your mood?”
After the development of an e-survey according to the Checklist for Reporting Results of Internet e-Surveys guidelines [16], investigators (S.C. and C.HK.P.) tested its usability and technical functionality before its implementation. The study protocol was approved by the Institutional Review Board (IRB) of the Asan Medical Center (2021-1490), and the obligation to obtain written informed consent was waived by the IRB.
The sample size estimation was done in terms of 30 participants per cell [17]. We allocated 30–40 samples for 10 cells (two groups of biological sex × five age groups), and the responses of 400 participants were collected from the 14 million general population panels of the survey company. The company sent emails for study enrollment to 1,000–2,000 panelists, and 400 participants among 599 panelists who accessed the survey system were finally enrolled. The company delivered the collected data to the investigators after excluding all identifiable personal information.
Measures
Coronavirus Reassurance-Seeking Behaviors Scale
The CRBS is a self-report rating scale which was developed to measure reassurance-seeking behaviors related to COVID-19 infection [14]. Based on the illness anxiety disorder (formerly known as hypochondriasis) model [6], fears of being sick may make a person anxious, thus inducing reassurance-seeking behavior. Relief from reassurance strengthens the original fear and increases preoccupation with the illness. In this COVID-19 pandemic, recurrent engagement in reassurance-seeking behaviors, such as checking bodily sensations, hand hygiene, visits to the doctor, or repeated media research, paradoxically increases anxiety. The CRBS consists of five items of reassurance-seeking behaviors which can be rated from 0 (not at all) to 4 (nearly every day), with reference to their expression over 2 weeks prior to response to the scale. Total scores range from 0 to 20, and a high score reflects high engagement in reassurance-seeking behavior. The CRBS has strong internal consistency reliability (α=0.90) [14]. In addition, an important problem in using the CRBS in Korean is the accuracy of the translation. In order to verify the accuracy of the translation, the original English version of the CRBS was translated into Korean by a bilingual expert, and the Korean version was translated back into English to ascertain accuracy.
Stress and Anxiety to Viral Epidemics-6 items
The Stress and Anxiety to Viral Epidemics-6 items (SAVE-6) scale is a self-rating scale developed to measure viral anxiety [18]. It was derived from the SAVE-9 scale [19]. It consists of six items which are rated on a 5-point Likert scale ranging from 0 (never) to 4 (always). A high total score reflects a severe degree of viral anxiety. In this study, we applied the original Korean version of the SAVE-6, and Cronbach’s alpha among this sample was 0.805.
Generalized Anxiety Disorder-7 items
The Generalized Anxiety Disorder-7 items (GAD-7) is a self-report rating scale for measuring the severity of general anxiety [20]. It consists of seven items that can be rated on a 4-point Likert scale from 0 (not at all) to 3 (nearly every day). A high total score reflects a severe degree of generalized anxiety. In this study, we applied the Korean version of GAD-7 [21], and Cronbach’s alpha was 0.948 in this sample.
Patient Health Questionnaire-9 items
The Patient Health Questionnaire-9 items (PHQ-9) is a selfrating scale which was developed for measuring the severity of depression [22]. It consists of nine items that can be rated on a 4-point Likert scale from 0 (not at all) to 3 (nearly every day). A higher total score reflects a severe degree of depression. In this study, we applied the Korean version of PHQ-9 [21], and Cronbach’ alpha was 0.930 in this sample.
Statistical analysis
In this study, we utilized IBM SPSS v26 (IBM Corp., Armonk, NY, USA), RStudio (RStudio, Boston, MA, USA), JASP (JASP, Amsterdam, The Netherlands) and Microsoft Office Excel 365 for data management. Descriptive statistics (percentages, mean, and standard deviation [SD]) were carried out to examine the distribution of responses. The reliability and validity of the Korean version of the CRBS were assessed utilizing both classical and item response theory (IRT). Both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) were carried out to explore and confirm the factor structure of the scale. Multigroup CFA was run to assess the measurement invariance of the scale across the variables of sex, depression, generalized anxiety, and viral anxiety. Item analysis was run to estimate the corrected item-total correlation, internal consistency reliability, and average inter-item correlation. The graded response model (GRM) was created in accordance with IRT. Slope and threshold parameters were estimated for the GRM. Moreover, IRT reliability and rho coefficient were calculated. Pearson’s product-moment correlation was run to estimate the association between PHQ-9, GAD-7, and SAVE-6.
RESULTS
All 400 participants were residents of Seoul (n=117, 29.3%), Pusan (n=30, 7.5%), Daegu (n=18, 4.5%), Daejeon (n=9, 2.3%), Gwangju (n=10, 2.5%), Incheon (n=31, 7.8%), Ulsan (n=8, 2.0%), Gyeonggi Province (n=108, 27.0%), Chungcheong Province (n=20, 5.0%), Jeolla Province (n=11, 2.8%), Gyeongsang Province (n=29, 7.3%), Gangwon Province (n=7, 1.8%), and Jeju Province (n=2, 0.5%). Demographic characteristics, responses to COVID-19 questions, and rating scale scores are presented in Table 1.
Clinical characteristics of the study subjects (N=400)
Initial exploratory factor analysis
Table 2 shows the item-level properties of the Korean version of the CRBS. The Kaiser–Meyer–Olkin value (0.799) and Bartlett’s Test of sphericity value (p<0.001) (Table 2) suggested the sampling adequacy and data suitability for conducting EFA. The corrected item-total correlation values of all items (ranging between 0.577 and 0.777) were above the minimum cut-off value (0.30). The EFA analysis showed the single factor model of the Korean version of the CRBS (eigenvalue=3.397). All items loaded highly on the explored factor (ranging between 0.0704 and 0.878) (Table 3).
Scale-level psychometric properties of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale
Item properties of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale
Confirmatory factor analysis
CFA results showed a good fit for the CRBS model (χ2=5.475, df=5, p value=0.361, χ2/df=1.095, comparative fit index [CFI]=0.999, Tucker–Lewis index [TLI]=0.998, root mean square error of approximation [RMSEA] [90% confidence interval, CI] [p value]=0.015 [0.000–0.073] [0.786], standardized root mean square residual [SRMR]=0.051) (Table 2). Factor loadings in CFA were good. They ranged between 0.619 and 0.849 (Table 3 and Figure 1). Multigroup CFA results showed that the Korean version of the CRBS measured reassuranceseeking behavior consistently across sex (CFI=1.000, TLI=1.011, RMSEA=0.000, SRMR=0.054), depression (PHQ-9 ≥10) (CFI=1.000, TLI=1.097, RMSEA=0.000, root mean square residual [RSMR]=0.063), general anxiety (GAD-7 ≥10) (CFI=1.000, TLI=1.001, RMSEA= 0.000, RSMR=0.061), and viral anxiety (SAVE-6 ≥15) (CFI= 1.000, TLI=1.008, RMSEA=0.000, RSMR=0.064).
Factor structure of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale (CRBS).
Graded response model analysis
Table 4 presents the GRM parameters (slope and threshold parameters) of the Korean version of the CRBS. Table 4 demonstrates that all the items have very high slope parameters (ranged between 1.925 and 4.493; mean=3.531). Among the items, item 3 has the highest discrimination ability and item 1 has the lowest discrimination ability. Overall, these items are highly efficient and able to provide good information about the latent trait assessed by the Korean version of the CRBS (Figure 2). Regarding threshold parameters, item 5 is the most difficult item among all the items, then item 3 and 4. In items 3, 4, and 5, all threshold coefficients are positive and suggest that above average levels of latent traits are required to endorse all the Likert-type response options. The scale information curve (Figure 3) shows that the Korean version of the CRBS provides more information about people between the 0.0 and 3.0 θ level.
Item fit, slope and threshold parameters of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale
Item response category characteristic curve for each item of Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale.
Scale information curve of the Korean version of the Coronavirus Reassurance-Seeking Behaviors Scale. SE, standard errror.
Reliability of the Korean version of Coronavirus Reassurance-Seeking Behaviors Scale and evidence based on relations to other variables
Scale-level psychometric properties were shown in Table 2. The mean item-total correlation (0.595) of the CRBS was above the recommended range (between 0.15 and 0.50). We checked that inter-item correlations and none of the correlations between the items were too high, revealing an absence of multicollinearity. The internal consistency reliability was good (Cronbach’s alpha=0.867, McDonald’s omega=0.871, and split-half reliability=0.911). The standard error of measurement (1.44) was below the cut-off (smaller than SD, 3.98/2). Furthermore, this scale had good IRT reliability (0.812) and a Rho coefficient (0.879). The Korean version of the CRBS showed a good convergent validity with SAVE-6 (r=0.431; 95% CI, 0.347–0.507; p<0.001), GAD-7 (r=0.574; 95% CI, 0.505–0.636; p<0.001), and PHQ-9 (r=0.575; 95% CI, 0.506–0.637; p<0.001). Moreover, the CRBS total score was significantly higher among those with depression (PHQ-9 ≥10, t[398]=10.36, p<0.001), those with general anxiety (GAD-7 ≥10, t[398]=11.14, p<0.001), and those with viral anxiety (SAVE-6 ≥15, t[398]=8.33, p<0.001).
DISCUSSION
In this study, we explored the reliability and validity of the Korean version of the CRBS among the general population during this COVID-19 pandemic. CFA results showed a good fit with the CRBS model, and a multigroup CFA revealed that it measures reassurance-seeking behavior consistently across variables of sex, depression, general anxiety, and viral anxiety. All the items have very high slope parameters, and the scale information curve shows that the Korean version of the CRBS provides more information about people between the 0.0 and 3.0 θ level. The internal consistency reliability was good, and it showed a good convergent reliability with other existing rating scales.
We observed that the Korean version of the CRBS is a valid and reliable rating scale which can be applied to measure reassurance-seeking behavior in this COVID-19 pandemic. Item analysis indicated that in terms of both classical test theory (corrected item-total correlation) and modern test theory (slope parameter of the GRM), this scale is able to discriminate reasonably between low scorers and high scorers. Threshold parameters of the GRM also show that items are comparatively difficult. Above average levels of a latent trait are needed to endorse at least the lowest response option (nearly every day over the last 2 weeks). The scale’s information curve provides a clearer picture of its efficiency in assessing the Korean people’s coronavirus reassurance-seeking behaviors. This scale is accurate at assessing reassurance-seeking behaviors of people between 0.0 and 3.0 θ level. Psychometric information related to IRT indicated the superiority of this scale in comparison to previously published studies reporting on the psychometric properties of the CRBS [14,23].
In this study, we observed good internal consistency in the Korean version of the CRBS, in accordance with previous studies [14,23]. Additionally, there was good convergent validity with another viral anxiety scale (SAVE-6) and the generalized anxiety scale (GAD-7). Since the CRBS is a rating scale measuring reassurance-seeking behavior specific to the viral epidemic, we explored the convergent validity of the CRBS with a nonspecific anxiety scale and an anxiety scale which was specific to the viral epidemic. The CRBS showed good convergent validity with the GAD-7 and SAVE-6 scales. The SAVE-6 was developed to measure an anxiety response specific to viral epidemics. It has already been validated among the general population [18,24] and special populations such as medical students, public workers, and cancer patients [25-27]. Results of this study support the usefulness of the CRBS in examining the mechanism of the association between viral anxiety and reassurance-seeking behaviors in relation to viral epidemics.
This study has some limitations. First, we could not find a reassurance-seeking scale that was not specific to the viral epidemic, as a measure of convergent validity of the CRBS. Although we applied the anxiety scale specific to viral epidemics, it would have been more informative if we could have used another reassurance-seeking behavior scale which was not specific to a viral epidemic. Second, we conducted this study using a professional online survey company. This may have led to bias, since the use of responses generated from panelists already registered in the survey system may have impacted results. This could have been avoided if the participants had been absolutely anonymous. Furthermore, self-report responses may evoke biases such as those of social desirability, and may result in the reluctance of participants to report their reassurance-seeking behaviors. Finally, 91.5% of the participants were already vaccinated, a factor that may have also influenced the results.
In conclusion, we confirm the Korean version of the CRBS as a psychometrically sound tool. It is a valid and reliable rating scale that measures reassurance-seeking behaviors associated with viral epidemics. As a model of anxiety disorder in relation to illness, it will assist with the examinations of people’s viral anxiety and reassurance-seeking behaviors in this COVID-19 pandemic.
Notes
Availability of Data and Material
The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.
Conflicts of Interest
Seockhoon Chung, a contributing editor of the Psychiatry Investigation, was not involved in the editorial evaluation or decision to publish this article. All remaining authors have declared no conflicts of interest.
Author Contributions
Conceptualization: Seockhoon Chung, Oli Ahmed. Data curation: Seockhoon Chung, C. Hyung Keun Park. Formal analysis: Seockhoon Chung, Oli Ahmed. Investigation: Seockhoon Chung. Methodology: Seockhoon Chung. Software: Changnam Kim. Supervision: Seockhoon Chung. Validation: Changnam Kim. Writing—original draft: all authors. Writing—review & editing: all authors.
Funding Statement
None