Psychiatry Investig Search

CLOSE


Psychiatry Investig > Volume 19(2); 2022 > Article
Kim, Kim, and Kim: Psychometric Properties of the Korean version of the Emotion Regulation Questionnaire (K-ERQ) in a Clinical Sample

Abstract

Objective

The Emotion Regulation Questionnaire (ERQ) is one of the widely used instruments to assess emotion regulation skills in many countries, including Korea. However, its psychometric properties have not been validated within this population. Also, the ERQ has increasingly been used in studies with psychiatric patients despite a general lack of validation in clinical settings. Therefore, the present study aimed to investigate the psychometric properties of the Korean version of the ERQ (K-ERQ) using a clinical sample in Korea.

Methods

One hundred and ninety-three psychiatric patients completed a packet of self-report measures, including K-ERQ, K-BDI-II, K-ASI-3, PCL-5-K, AUDIT-K. Confirmatory factor analysis (CFA) was administered to investigate the factor structure of the K-ERQ, and internal reliability and validity were examined.

Results

Results of the CFA supported the two-factor structure, but only after the removal of one item. The K-ERQ showed good internal consistency reliability, and its concurrent validity was also confirmed. Cognitive reappraisal was negatively correlated with depression and alcohol use disorder-related symptoms, and expressive suppression was positively correlated with depression, anxiety sensitivity, posttraumatic stress disorder (PTSD)-related symptoms and alcohol use disorder-related symptoms. Significant group differences were found in the use of emotion regulation strategies; patients with PTSD reported the higher level of cognitive reappraisal than patients with depressive disorders, bipolar disorders, and attention-deficit hyperactivity disorder.

Conclusion

The 9-itemed K-ERQ is a reliable and valid tool to assess the emotion regulation strategies in a Korean clinical sample. Our study also adds preliminary evidence on the usefulness of the ERQ in clinical settings.

INTRODUCTION

Emotion is a crucial factor that allows individuals to pay attention to external stimuli, adjust decision-making, and shape their behavior. It is also very beneficial in coordinating social interactions. However, emotions can be problematic when the type, intensity, and duration are inappropriate [1]. Therefore, appropriate emotion regulation is central to human adaptation. Previous studies have confirmed that the ability to regulate emotions successfully is critical to many domains of functioning. In contrast, difficulties in emotion regulation act as a key risk factor for psychopathological symptoms [2]. Thus, accurate assessment and intervention for emotion regulation in clinical settings are essential, and developing and validating reliable measures of emotion regulation is required.
According to Gross, emotion regulation refers to the “processes by which individuals influence which emotions they have, when they have them, and how they experience and express these emotions.” [3] In his process model of emotion, emotion regulation is conceptualized as selecting, modifying, paying attention, and appraising situations to yield particular responses over time [4,5]. One of the most widely adopted scales for emotion regulation is the Emotion Regulation Questionnaire (ERQ) [4], which is based on the model by Gross [5]. The ERQ addresses two main strategies of emotion regulation; cognitive reappraisal and expressive suppression. Cognitive reappraisal is a form of cognitive change and is among the antecedent-focused emotion regulation process. It involves reevaluating the emotion-eliciting situation to regulate the emotional impact. Cognitive reappraisal is often used to decrease negative emotions and is associated with positive mental health [6]. Whereas expressive suppression is a response modulation and refers to the inhibition of emotion-expression behavior after generating an emotional response. Expressive suppression is known to be associated with poor memory and less positive relations with others [7]. The ERQ consists of 10 items and has been validated in several languages and cultures including, Italy, Spain, and Taiwan [8-10]. Its brevity, and excellent reliability and validity, make the questionnaire one of the most widely used tools in measuring emotion regulation.
Unfortunately, although the ERQ has also been commonly used within the Korean population [11,12], its psychometric properties have not been investigated yet. The lack of cross-cultural validation is problematic, especially when cultural values play a critical role in the experience of emotions and the employment of specific regulation strategies. Indeed, a few validation studies that have been administered in other countries have failed to replicate the original structure of the two independent factors in Gross and John [4]. For instance, the original ERQ factor structure could not be confirmed using a community sample in Australia and the United Kingdom [13]. The researchers found that two items showed high covariance with each other and proposed a 9-item version of the ERQ by removing one item. Another comparable finding was also observed in a study investigating the psychometric properties of the ERQ using an Italian undergraduate and community sample [14]. They found a poor model fit when testing the original factor structure and consequently removed two items from the scale as the items had high error covariance. Furthermore, Wiltink and Glaesmer [15] demonstrated that the factor structure of the original ERQ could not be supported in a German community sample as the result of the confirmatory factor analysis showed inadequate fit, with one item from the cognitive reappraisal scale loading onto both factors (cognitive reappraisal and expressive suppression). Therefore, to achieve a good model fit, they modified the model so that the one item could cross-load on both factors. Overall, this discrepancy in factor structure demands cross-cultural evaluation, and there is currently limited evidence on the tool’s utility with the Korean population.
Besides the potential cultural differences, other factors may contribute to the variation in the factor structure between the samples. Such as, in regards to the low fit indices of confirmatory factor analysis obtained in one Italian sample [16], the authors suggest that this may be due to the methodological issues associated with the translation of the items. More specifically, they addressed that the Italian translation of the ERQ provides more examples that allow the respondents to distinguish between positive and negative emotions better than the German version of the ERQ.
Another noteworthy point is that even though the original scale was developed using a non-clinical sample, an increasing number of studies have utilized the ERQ to examine the specific impairment in emotion regulation within different mental disorders [17-20]. Recent studies using the ERQ include individuals with experience of trauma or posttraumatic stress disorder symptoms [21-24], eating disorders [25,26], bipolar disorders [27], social anxiety disorder [28], and obsessive-compulsive disorder [29]. However, the psychometric properties of the ERQ in the clinical setting have not been fully verified yet. To the best of our knowledge, only one study has examined the factor structure of the ERQ using a clinical sample. Marco et al. [30] analyzed the psychometric properties of the Spanish version of the ERQ in participants with personality disorder, with the majority having a borderline personality disorder. Although this study demonstrates the instruments’ validity outside the non-clinical population, it is limited in that the sample is focused narrowly on personality disorders, providing only partial empirical support for the use of ERQ in clinical settings.
The ERQ is also commonly used in clinical and research contexts in Korea as one of the primary instruments in assessing emotion regulation [11,31-35]. Studies utilizing the Korean version of the ERQ in a clinical sample include investigating the mental health of youths with traumatic experience [31], emotion regulation of patients with bipolar disorder in relation to their working memory [32], and the characteristics of adolescents diagnosed with depression [33]. There are also studies involving non-clinical samples. These include research examining the cultural and gender differences in emotion regulation between Korean and American college students [11], emotion regulation of male adults in association with their brain activity [34], and the contribution of emotion regulation on attentional processing of healthy adults [35].
In summary, considering the ERQ’s common usage with psychiatric patients internationally and the heterogeneity between community and clinical sample on emotion regulation [19,36], it is imperative to test the reliability and validity of the ERQ in the clinical population. The present study investigates the psychometric properties of the Korean version of the ERQ (K-ERQ) and evaluates whether the two-factor structure is also replicated in the Korean population. In particular, we were interested in individuals with mental disorders as there is limited data on whether the assessment is also suitable outside the community and the student sample. Therefore, the current study explored the factor structure, internal consistency, and concurrent validity of the K-ERQ using a sample of psychiatric patients.

METHODS

Participants

This study was approved by the Institutional Review Boards of the corresponding author’s medical center (HYUH 2021-03-058-004). The data were obtained from patients who visited the psychiatric department of a university medical center from February 2020 to March 2021. Our sample consisted of 193 patients with an average age of 29.90 (age range=18 to 79, SD=13.58), and 69.4% of the total sample were male. Most participants (96.4%) have at least graduated from high school as 53.9% of the participants reported a high school diploma as their highest degree of education, and this was followed by a college degree (23.3%), associate degree (13.0%), graduate degree (6.2%), and middle school diploma or lower (3.6%). Primary diagnoses were made using a Structured Clinical Interview for DSM-5 [37]. The majority of the patients had multiple diagnoses, and the average number of diagnoses was 2.00 (SD=0.93). The exclusion criteria were current diagnosis of 1) mood disorders with psychotic features, 2) schizophrenia spectrum disorders, 3) intellectual disability, or 4) neurocognitive disorders. The detailed sample characteristics are presented in Table 1.
Among the mood disorder subgroups, the majority of the patients met the criteria for a current episode. Within the major depressive disorder (MDD) group, 4 participants were in partial remission while 67 participants had a current episode of MDD and the severity of their episode were as follows: mild (n=7), moderate (n=43), and severe (n=17). For persistent depressive disorder (PDD) group, the subtypes are as follows: with pure dysthymic syndrome (n=1), with persistent major depressive episode (n=5), with intermittent major depressive episodes, with current episode (n=13), and with intermittent major depressive episodes, without current episode (n=2). There were 3 participants with bipolar I disorder and 16 participants with bipolar II disorder in the bipolar disorder group (n=19). For patients with bipolar I disorder, the subtypes were: current or most recent episode depressed, moderate (n=1), current or most recent episode manic, mild (n=1), current or most recent episode manic, in full remission (n=1). Lastly, for patients with bipolar II disorder, the subtypes were: current or most recent episode depressed, moderate (n=9), current or most recent episode depressed, severe (n=2), current or most recent episode depressed, in partial remission (n=2), current or most recent episode hypomanic, severe (n=2), and current or most recent episode hypomanic, in full remission (n=1).

Materials

The Korean version of the Emotion Regulation Questionnaire (K-ERQ)

The ERQ [4] is a 10-item measure that assesses the respondent’s habitual use of cognitive reappraisal and expressive suppression. The cognitive reappraisal subscale consists of six items and contains questions such as “When I want to feel more positive emotion (such as joy or amusement), I change what I’m thinking about.” The expressive suppression subscale is four items and includes questions such as “I keep my emotions to myself.” Both subscales require participants to evaluate on a 7-point Likert-type scale (1=strongly disagree, 7=strongly agree). The present study utilized the K-ERQ translated by Shon [38]. This translation is the widely used version of the ERQ by many researchers and clinicians in Korea (for example, Oh et al. [32] and Lee and Jang [39]). The internal reliability in Shon’s [38] study for cognitive reappraisal and expressive suppression subscale was 0.85 and 0.73, respectively.

The Korean version of the Beck Depression Inventory-II (K-BDI-II)

The BDI-II [40] is a 21-item questionnaire that measures the severity of self-reported depressive symptoms within the past two weeks. It reflects the major depressive disorder criteria in the Diagnostic and Statistical Manual of Mental DisordersFourth Edition (DSM-IV). Respondents are asked to rate each item on a 4-point Likert type scale ranging from 0 to 3, with higher scores indicating more severe depressive symptoms. The Korean version of the BDI-II was validated by Lim et al. [41], showing good internal reliability (Cronbach α=0.89) and test-retest reliability of 0.90.

The Korean version of the Anxiety Sensitivity Index-3 (K-ASI-3)

The Anxiety Sensitivity Index-3 [42] is an 18-item self-report instrument assessing three dimensions of anxiety sensitivity; physical, cognitive, and social concerns. Each item is rated on a 5-point Likert type scale ranging from 0 to 4. The internal consistency (α) of the Korean version of the ASI-3 was 0.87 [43].

The Korean version of the Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5-K)

The PCL-5 [44] assess the posttraumatic stress disorder (PTSD)-related symptoms, and the Korean version of PCL-5 was validated by Kim et al. [45]. It consists of 20 items that correspond to the criteria of PTSD in the Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5), and the respondent rate each item using a 5-point Likert type scale. The Cronbach’s alpha of the Korean version of the PCL-5 for reexperiencing, avoidance, negative alterations in cognitions and mood, and hyperarousal subscale were 0.92, 0.91, 0.93, and 0.93, respectively [45].

The Korean version of the Alcohol Use Disorders Identification Test (AUDIT-K)

AUDIT is a brief 10-item measure with a 5-point Likert-type scale that aims to identify individuals with alcohol problems. The Korean version of the AUDIT was validated by Joe et al. [46] The psychometric properties of the Korean version showed excellent test-retest reliability of 0.93.

Data analysis plan

Prior to statistical analyses, the normality of the data was examined by calculating the skewness and kurtosis of the K-ERQ. The skewness values of the K-ERQ items ranged from -0.38 to 0.35, and the kurtosis values ranged from -1.16 to -0.90, suggesting a normal distribution according to the limits recommended by Kline [47].
Confirmatory factor analysis (CFA) using maximum likelihood estimation was conducted using Mplus version 7.0 (Muthén & Muthén, Los Angeles, CA, USA) [48]. First, we tested the two-factor structure by Gross and John [4], where cognitive reappraisal factor consisted of six items (item 1, 3, 5, 7, 8, 10) and expressive suppression factor consisted of four items (item 2, 4, 6, 9). The two latent factors were allowed to correlate freely. The factor structure was evaluated using the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the root-mean-square error of approximation (RMSEA), and the standardized root-mean-square residual (SRMR). The recommended cut-off values in evaluating the factor structure are as follows. CFI and TLI values over 0.90, SRMR under 0.08 [49], and RMSEA value under 0.08 represent an acceptable fit [50]. In case of a poor model fit, modification indices (MI) and factor loadings were used as a reference to propose alternative models, and these models were compared to the original model to determine the one with the best fit to the data. Then, the model with the best fit was used for subsequent analyses.
The descriptive statistics of the sample, internal consistency, t-test analysis, one-way ANOVA, and correlation analysis were carried out using SPSS version 22.0 (IBM Corp., Armonk, NY, USA) [51]. The internal consistency of the K-ERQ was assessed using Cronbach’s alpha estimates. Independent sample t-test analysis was performed to explore the gender differences between the two subscales of the K-ERQ. One-way ANOVA was used to examine the differences in the two subscales of the K-ERQ between the diagnostic groups. These diagnostic groups consisted of five groups, which were made based on the most prevalent mental health disorders found in our patient sample (MDD, PDD, bipolar disorders [BDs], attentiondeficit/hyperactivity disorder [ADHD], and PTSD). Pearson’s correlations were calculated between K-ERQ, K-BDI-II, K-ASI-3, PCL-5-K, and AUDIT-K to examine the relationship between K-ERQ and psychiatric symptomatology that have been confirmed in previous studies, and a p-value of less than 0.05 was considered statistically significant.

RESULTS

Confirmatory factor analysis

The CFA results suggest that the two-factor structure by Gross and John [4] is not supported within the Korean clinical sample. The fit indices did not meet the recommended cutoff values, indicating a poor model fit (CFI=0.89, TLI=0.85, RMSEA=0.10, SRMR=0.06). We examined the modification indices (MI), and high covariance was found between item 1 and item 7 (MI=27.65, parameter change=0.81). Both items belong in the cognitive reappraisal subscale, and the two items are very similar (Item 1=When I want to feel more positive emotion [such as joy or amusement], I change the way I’m thinking about. / Item 7=When I want to feel more positive emotion, I change the way I’m thinking about the situation.). Subsequently, alternative models were proposed for comparison to determine the model with the best fit to the data. Three models were compared: 1) model 1: original two-factor model by Gross and John [4], 2) model 2: modified two-factor model excluding item 1, 3) model 3: modified two-factor model allowing covariance between item 1 and 7. For model 2, item 1 was removed as item 7 loaded more strongly to the cognitive reappraisal subscale (item 1=0.59, item 7=0.63), and the two items showed a high degree of redundancy in terms of their content.
The fit indices for all three models were computed (Table 2). The model 2, the modified two-factor model excluding item 1, provided the best fit to the data: χ2=48.23, p<0.01, CFI=0.96, TLI=0.94, RMSEA=0.07, SRMR=0.05. The nine items showed significant and salient factor loadings ranging from 0.43 to 0.83 (Figure 1). Therefore, model 2 was selected for further analysis.

Reliability

The Cronbach’s alpha was calculated to evaluate the internal consistency of the K-ERQ. In our sample, Cronbach’s alpha was 0.82 for cognitive reappraisal and 0.75 for expressive suppression.

Differences between gender and diagnostic groups

To examine the gender difference in the emotion regulation strategies measured by the K-ERQ, independent sample t-test analysis was conducted. The results indicated no significant gender difference in the K-ERQ total score (t=0.09, p= 0.93). Similarly, there was also no gender differences in each of the subscales (cognitive reappraisal, t=-0.12, p=0.91; expressive suppression, t=0.33, p=0.75).
One-way ANOVA and Scheffe’s post-hoc analysis were used to compare the use of specific emotion regulation strategies between diagnostic groups (Table 3). Excluding the groups categorized as other diagnoses, five diagnostic groups (MDD, PDD, BD, ADHD, PTSD) were examined for any differences in the K-ERQ factors. The results indicated significant differences in the cognitive reappraisal subscale among the five diagnostic groups. Scheffe’s post hoc analysis showed that patients with PTSD reported significantly higher scores than patients with PDD or ADHD. No significant group differences were found for the expressive suppression scale.

Concurrent validity

To ensure that the K-ERQ has concurrent validity, correlations with other psychiatric symptoms were examined as well. As a result, both scales showed significant correlations with the relevant psychiatric symptoms (Table 4). Cognitive reappraisal was negatively correlated with depression (r=-0.26, p<0.001) and alcohol use disorder-related symptoms (r=-0.14, p<0.05). Whereas expressive suppression was positively correlated with depression (r=0.25, p<0.001), anxiety sensitivity (r=0.20, p<0.01), PTSD-related symptoms (r=0.21, p<0.01), and alcohol use disorder-related symptoms (r=0.15, p<0.05).

DISCUSSION

Our study aimed to examine the psychometric properties of the K-ERQ using a clinical sample. Our results indicated that a two-factor structure of the K-ERQ, comprised of nine items, provided the best fit to our data. This differs from the original two-factor structure, which has ten items suggested by Gross and John [4]. However, this slight discrepancy in factor structure has also been reported in other validation studies, and minor modifications were made to the original scale to better suit the intended population.
Specifically, the studies that have suggested an adjustment to the original scale include samples outside the student population. Such as, in a German validation with a large community sample [15], the original factor structure was not confirmed, with item 8 loading onto both factors. In two separate studies involving community samples [13,14], a high covariance was found between items 1 and 3, leading researchers to omit either one or two items from the original scale. Our findings also demonstrated an inadequate fit for the original structure, with two of the items showing high covariance (items 1 and 7). Therefore, in the present study, two alternative models were compared with the original two-factor model by Gross and John [4]. The two alternative models include a model that excludes one item (item 1) from the two items that showed high covariance (items 1 and 7) and another model that allows a covariance between the two items (items 1 and 7). The results showed that the model that discarded one item demonstrated the best model fit. Consequently, following the previous validation studies that have removed items from the original scale, we also altered the scale by eliminating one item from the pair of items (item 1 and 7) that showed high covariance for the following reasons.
First, one possible explanation for the high covariance is that the two items arguably ask the same aspect of emotion regulation (Item 1: When I want to feel more positive emotion [such as joy or amusement], I change what I’m thinking about. Item 7: When I want to feel more positive emotion, I change the way I’m thinking about the situation.). Items 1 and 7 in the cognitive reappraisal subscale share similar content and wording, with both items measuring the extent to which individuals change their thoughts to feel more positive. To some extent, including redundant items in a scale is favorable as they measure a construct in somewhat different ways, but nearly identical items hold a slight advantage [52]. Problems relating to the redundancy of the ERQ items are also evident in a study using an Australian and UK sample [13] as well as in another study involving an Italian sample [14], and these researchers have decided to discard one or two items from the scale. Likewise, we have also removed one item from the original scale to suggest a psychometrically improved version of the questionnaire specifically for the Korean clinical population. Future research might benefit from adopting item response theory, which can improve the accuracy of scores and allow researchers to increase the instrument’s efficiency by including only the discriminative items [53]. Perhaps, as some items on the ERQ are almost identical in their meaning, a further psychometric analysis might help determine and eliminate redundant items. After careful consideration from a statistical and theoretical perspective, item 1 was deleted from the K-ERQ in our study. The K-ERQ with nine items presented an excellent fit with salient loadings on each item.
The 9-itemed K-ERQ also presented good internal consistency reliability, and its concurrent validity was also confirmed through correlations with other measures as expected in the clinical population. Expressive suppression showed significant positive correlations with depression and anxiety sensitivity, and cognitive reappraisal showed significant negative correlations with depression. This is consistent with prior studies that have explored the psychometric properties of the ERQ in non-clinical populations [13,15,54]. These similar trends in correlation analyses among validation studies provide evidence that cognitive reappraisal is related to positive functioning and expressive suppression is related to negative functioning regardless of culture and diagnostic status. Moreover, previous studies have shown that cognitive reappraisal is a more effective strategy in regulating negative emotions [55]. While, in an experimental study, expressive suppression was associated with increased blood pressure during an emotionally taxing conversation [56]. Thus, our findings are generally in line with previous work involving non-clinical populations.
In addition to these measures of affective symptoms, we have also examined the associations between emotion regulation strategies and PTSD-related and alcohol use disorder-related symptoms. Alcohol use disorder-related symptoms had significant correlations with both subscales, while PTSD-related symptoms were associated with expressive suppression only. Further, the present study provides additional evidence on the two-factor model posited by Gross and John [4]. In this model, cognitive reappraisal and expressive suppression are independent constructs associated with different outcomes. Our results have also indicated that the subscales of the ERQ are two independent factors, and the two subscales have shown contrasting patterns when exploring their associations with various psychopathological symptoms. Future studies should also consider the contextual factors as they play a crucial role in an individual’s preference for emotion regulation. Indeed, unlike the first generation of studies that have broadly explored emotion regulation strategies in terms of their adaptiveness, the second generation of studies on emotion regulation has highlighted that the outcome of a specific strategy can vary depending on the context [57]. Thus, it would be worthwhile to explore the relationship between the two emotion regulation strategies and the relevant psychiatric symptoms in the context of everyday life.
We also examined the differences between men and women in their use of emotion regulation strategies. Previous studies involving non-clinical samples have reported that men tend to suppress their emotions more frequently than women [4,58,59]. However, the results of our study indicate that no notable difference exists between men and women in regard to their habitual use of expressive suppression. This is in line with the prior findings of Campbell-Sills et al. [60] that, contrary to the control group, no gender difference was found in the suppression of negative emotion in a clinical group involving patients with anxiety and mood disorders. Also, empirical evidence on patients with MDD suggests that depressed patients suppress their emotions more frequently than the healthy control group [61,62]. As a large percentage of our participants were diagnosed with depressive disorders, excessive use of suppression may be an essential characteristic of our sample that goes beyond the inherent gender differences in terms of emotion regulation.
There were also significant differences between the diagnostic groups regarding the cognitive reappraisal subscale. Patients with PTSD were more likely to employ cognitive reappraisal than patients with PDD or ADHD. Regarding this group difference, one previous study [63] showed that cognitive reappraisal was not associated with PTSD-related symptoms in a group of trauma-exposed individuals. In contrast, our study showed somewhat different results as this group exhibited the highest level in their use of cognitive reappraisal. In addition, previous research has shown that patients with MDD exhibit increased suppression of emotions [61] and decreased use of cognitive reappraisal [62]. Concerning bipolar disorders, patients with bipolar I disorder endorsed significantly lower levels of cognitive reappraisal than healthy controls [32], and euthymic bipolar patients reported decreased use of cognitive reappraisal and increased use of expressive suppression compared to the control group [64]. Impairments in emotion regulation are also found in ADHD, with patients engaging in expressive suppression more frequently and cognitive reappraisal less frequently than the healthy controls [65,66]. Therefore, one possible explanation on why the PTSD group scored significantly higher on the cognitive reappraisal scale than few of the other diagnostic groups could be that patients with PTSD may not differ in the self-reported frequency of cognitive reappraisal when compared to the non-clinical group. However, they may exhibit less efficient usage, or their level of cognitive reappraisal combined with other types of emotion regulation could be associated with a negative outcome. Previous studies have noted that cognitive reappraisal may not be an adaptive regulation strategy per se. Factors such as contextual influences and other emotion regulation strategies should be considered when evaluating its adaptiveness. For instance, interaction between low emotional clarity and high cognitive reappraisal was associated with problematic cannabis use [67], and cognitive reappraisal was negatively associated with psychological functioning in the context of oppression for individuals of ethnic minority groups [68]. Moreover, considering that PTSD-related symptoms were significantly correlated with expressive suppression, but not with cognitive reappraisal, the key maladaptive emotion regulation strategy in the maintenance of PTSD could be avoidance or emotional suppression, as suggested by studies that revealed a significant relationship between PTSD symptoms and avoidance or suppression [69-71]. However, as no comparison between healthy controls was administered in the current study, future studies could benefit from recruiting clinical and non-clinical samples to examine the group differences and the combined effect of cognitive reappraisal with other emotion regulation strategies that contribute to their clinical symptoms.
Lastly, differences in the emotion regulation strategies depending on the mood states among patients with mood disorder could not be carried out due to the small size of our sample. Regarding this, research on the emotion regulation of participants currently or previously diagnosed with MDD and healthy controls have shown that patients currently diagnosed with depression exhibited less use of cognitive reappraisal than remitted patients with depression [72]. Therefore, additional research investigating the associations with mood states may help yield more information on the emotion regulation of the clinical population.

Limitations and conclusion

The present study has several limitations that should be taken into consideration. First, our sample is relatively small, and most of the participants were diagnosed with depressive disorders. This may limit the generalizability of our findings to other mental illnesses. Future studies should replicate the results of our research using a more diverse group. Second, the lack of a healthy control group makes it difficult to conclude whether our findings are limited to the clinical population. It would be interesting to include healthy subjects to examine whether the factor structure of nine items is stable across the Korean population. Third, test-retest reliability was not obtained in our study. Therefore, the temporal stability of the KERQ should be examined in future studies.
Nonetheless, the present study is the first, to our knowledge, to examine the psychometric properties of the Korean translation of the ERQ using a clinical sample. The 9-itemed K-ERQ is a reliable and valid tool for assessing the emotion regulation strategies of adults with mental disorders. As the psychometric properties of the ERQ in clinical samples have not been sufficiently investigated yet, our study adds to the growing literature by providing preliminary evidence on its usefulness in clinical settings.

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: all authors. Data curation: Kawon Kim. Formal analysis: Kawon Kim. Funding acquisition: Sojung Kim. Investigation: Kawon Kim, Sojung Kim. Methodology: Kawon Kim, Sojung Kim. Project administration: Sojung Kim. Supervision: Sojung Kim. Visualization: Kawon Kim. Writing—original draft: Kawon Kim. Writing—review & editing: Seok Hyeon Kim, Sojung Kim.

Funding Statement

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF2021S1A5A8068446).

ACKNOWLEDGEMENTS

The authors thank Ms. Yubeen Bae, Ms. Eunyoung Son, Ms. Euwon Joh, Ms. Yeonkyeong Son and Ms. Jiyoung Hong at the Hanyang University Medical Center, for their assistance in data collection.

Figure 1.
Standardized factor loadings of the 9-itemed K-ERQ. All factor loadings were statistically significant at p<0.001. K-ERQ, Korean version of the Emotion Regulation Questionnaire.
pi-2021-0269f1.jpg
Table 1.
Demographic characteristics
Characteristics Value
Age 29.90±13.58
Gender
 Male 134 (69.4)
 Female 59 (30.6)
Marital status
 Never-married 130 (67.4)
 Married 54 (28.0)
 Remarried 5 (2.6)
 Separated 1 (0.5)
 Divorced 2 (1.0)
 Widowed 1 (0.5)
Education
 Middle school diploma or lower 7 (3.6)
 High school diploma 104 (53.9)
 Associate degree 25 (13.0)
 College degree 45 (23.3)
 Graduate degree 12 (6.2)
Socioeconomic status (N=183)
 Low 30 (16.4)
 Low to middle 32 (17.5)
 Middle 78 (42.6)
 Middle to high 27 (14.8)
 High 16 (0.09)
Primary diagnoses
 Major depressive disorder 71 (36.8)
 Persistent depressive disorder 21 (10.9)
 Bipolar disorders 19 (9.9)
 Attention-deficit/hyperactivity disorder 14 (7.3)
 Posttraumatic stress disorder 12 (6.2)
 Other* 56 (28.9)

Data are presented as mean±standard deviation or N (%).

* other primary diagnoses include adjustment disorder, anxiety disorders, gambling disorder, obsessive compulsive disorder, alcohol use disorder, etc.

Table 2.
Model summary of the three models
Model χ2 df p CFI TLI RMSEA SRMR
Model 1 101.76 34 <0.001 0.89 0.85 0.10 0.06
Model 2 48.23 26 <0.01 0.96 0.94 0.07 0.05
Model 3 73.93 33 <0.001 0.93 0.91 0.08 0.05

Model 1: original two-factor model by Gross and John;4 Model 2: modified two-factor model without item 1; Model 3: modified two-factor model allowing covariance between item 1 and 7. CFI, comparative fit index; TLI, Tucker-Lewis index; RMSEA, rootmean-square error of approximation; SRMR, standardized rootmean-square residual

Table 3.
Differences between diagnostic groups
Diagnostic group N Mean SD ANOVA
Post-hoc (Scheffe’s test)
F p
K-ERQ reappraisal 3.24 0.01 e>b, e>d
 MDD (a) 71 19.06 6.21
 PDD (b) 21 17.29 7.53
 BD (c) 19 19.32 7.70
 ADHD (d) 14 17.36 7.13
 PTSD (e) 12 25.58 8.03
K-ERQ suppression 2.51 0.05
 MDD (a) 71 17.59 4.81
 PDD (b) 21 19.29 5.15
 BD (c) 19 14.32 6.28
 ADHD (d) 14 18.36 6.87
 PTSD (e) 12 19.08 6.49

SD, standard deviation; ANOVA, analysis of variance; K-ERQ, Korean version of the Emotion Regulation Questionnaire; MDD, major depressive disorder; PDD, persistent depressive disorder; BD, bipolar disorders; ADHD, attention-deficit/hyperactivity disorder; PTSD, posttraumatic stress disorder

Table 4.
Correlations between the 9-itemed K-ERQ and other measures
K-ERQ-CR K-ERQ-ES K-BDI-II K-ASI-3 PCL-5-K AUDIT-K
K-ERQ-CR -
K-ERQ-ES 0.17* -
K-BDI-II -0.26 0.25 -
K-ASI-3 -0.13 0.20 0.66 -
PCL-5-K -0.10 0.21 0.76 0.76 -
AUDIT-K -0.14* 0.15* 0.20 0.23 0.18* -
M (SD) 19.36 (7.01) 17.18 (5.59) 32.43 (13.41) 31.89 (17.99) 45.37 (19.10) 8.39 (9.21)

* p<0.05;

p<0.01;

p<0.001.

K-ERQ-CR, Korean version of the Emotion Regulation Questionnaire-Cognitive Reappraisal; K-ERQ-ES, Korean version of the Emotion Regulation Questionnaire-Expressive Suppression; K-BDI-II, Korean version of the Beck Depression Inventory-II; K-ASI-3, Korean version of the Anxiety Sensitivity Index-3; PCL-5-K, Korean version of the Posttraumatic Stress Disorder Checklist for DSM-5; AUDIT-K, Korean version of the Alcohol Use Disorder Identification Test

REFERENCES

1. Gross JJ. Emotion Regulation: Conceptual and Empirical Foundations. In: Gross JJ editors. Handbook of Emotion Regulation. 2nd Ed. New York, NY: Guilford Publications; 2014. p.3-20.

2. Cole PM, Michel MK, O’ L, Teti D. The Development of Emotion Regulation and Dysregulation: A Clinical Perspective. In: Fox NA editors. The Development of Emotion Regulation: Biological and Behavioral Considerations. Chicago, IL: University of Chicago Press; 1994. p.73-100.

3. Gross JJ. The emerging field of emotion regulation: An integrative review. Rev Gen Psychol 1998;2:271-299.
crossref
4. Gross JJ, John OP. Individual differences in two emotion regulation processes: implications for affect, relationships, and well-being. J Pers Soc Psychol 2003;85:348-362.
crossref pmid
5. Gross JJ. Antecedent and response focused emotion regulation: divergent consequences for experience, expression, and physiology. J Pers Soc Psychol 1998;74:224-237.
crossref pmid
6. Hu T, Zhang D, Wang J, Mistry R, Ran G, Wang X. Relation between emotion regulation and mental health: a meta-analysis review. Psychol Rep 2014;114:341-362.
crossref pmid
7. Richards JM, Gross JJ. Personality and emotional memory: how regulating emotion impairs memory for emotional events. J Res Pers 2006;40:631-651.
crossref
8. Balzarotti S, John OP, Gross JJ. An Italian adaptation of the emotion regulation questionnaire. Eur J Psychol Assess 2010;26:61-67.
crossref
9. Cabello R, Salguero JM, Fernández-Berrocal P, Gross JJ. A Spanish adaptation of the Emotion Regulation Questionnaire. Eur J Psychol Assess 2013;29:234-240.
crossref
10. Li CH, Wu JJ. Psychometric evaluation of the Chinese version of the Emotion Regulation Questionnaire in Taiwanese college students. Assessment 2020;27:1300-1309.
crossref pmid
11. Kwon H, Yoon KL, Joormann J, Kwon JH. Cultural and gender differences in emotion regulation: relation to depression. Cogn Emot 2013;27:769-782.
crossref pmid
12. Yang E, Ha J. The mediating effect on perfectionistic self-presentation between emotion suppression and interpersonal problems in university students. Korean J Youth Stud 2020;27:379-403.
crossref
13. Spaapen DL, Waters F, Brummer L, Stopa L, Bucks RS. The Emotion Regulation Questionnaire: validation of the ERQ-9 in two community samples. Psychol Assess 2014;26:46-54.
crossref pmid
14. Balzarotti S. The emotion regulation questionnaire: factor structure and measurement invariance in an Italian sample of community dwelling adults. Curr Psychol 2019;40:1-12.
crossref
15. Wiltink J, Glaesmer H. Regulation of emotions in the community: suppression and reappraisal strategies and its psychometric properties. GMS Psycho Soc Med 2011;8:1-12.

16. Sala MN, Molina P, Abler B, Kessler H, Vanbrabant L, van de Schoot R. Measurement invariance of the Emotion Regulation Questionnaire (ERQ). A cross-national validity study. Eur J Dev Psychol 2012;9:751-757.
crossref
17. Fernando SC, Beblo T, Schlosser N, Terfehr K, Otte C, Löwe B, et al. The impact of self-reported childhood trauma on emotion regulation in borderline personality disorder and major depression. J Trauma Assoc 2014;15:384-401.
crossref
18. Blalock DV, Kashdan TB, Farmer AS. Trait and daily emotion regulation in social anxiety disorder. Cogn Ther Res 2016;40:416-425.
crossref
19. Fitzgerald JM, MacNamara A, Kennedy AE, Rabinak CA, Rauch SA, Liberzon I, et al. Individual differences in cognitive reappraisal use and emotion regulatory brain function in combat-exposed veterans with and without PTSD. Depress Anxiety 2017;34:79-88.
crossref pmid
20. Moran EK, Culbreth AJ, Barch DM. Emotion regulation predicts everyday emotion experience and social function in schizophrenia. Clin Psychol Sci 2018;6:271-279.
crossref pmid
21. Hannan SM, Orcutt HK. Emotion regulation in undergraduate students with posttraumatic stress symptoms: a multimethod study. Psychol Trauma 2020;12:643-650.
crossref pmid
22. Khan AJ, Maguen S, Straus LD, Nelyan TC, Gross JJ, Cohen BE. Expressive suppression and cognitive reappraisal in veterans with PTSD: results from the mind your heart study. J Affect Disord 2021;283:278-284.
crossref pmid
23. Nagulendran A, Jobson L. Exploring cultural differences in the use of emotion regulation strategies in posttraumatic stress disorder. Eur J Psychotraumatol 2020;11:1-11.
crossref
24. Post LM, Youngstrom E, Connell AM, Zoellner LA, Feeny NC. Transdiagnostic emotion regulation processes explain how emotion-related factors affect co-occurring PTSD and MDD in relation to trauma. J Anxiety Disord 2021;78:102367.
crossref pmid
25. Naumann E, Svaldi J. Influence of suppression and reappraisal on eating-related symptoms and ruminative thinking in anorexia nervosa and bulimia nervosa. Behav Res Ther 2021;141:103851.
crossref pmid
26. Meule A, Richard A, Schnepper R, Reichenberger J, Georgii C, Naab S, et al. Emotion regulation and emotional eating in anorexia nervosa and bulimia nervosa. Eat Disord 2021;29:175-191.
crossref
27. Aslan IH, Baldwin DS. Ruminations and their correlates in depressive episodes: between-group comparison in patients with unipolar or bipolar depression and healthy controls. J Affect Disord 2021;280:1-6.
crossref
28. Kivity Y, Cohen L, Weiss M, Elizur J, Huppert JD. The role of expressive suppression and cognitive reappraisal in cognitive behavioral therapy for social anxiety disorder: a study of self-report, subjective, and electrocortical measures. J Affect Disord 2021;279:334-342.
crossref pmid
29. Ferreira S, Couto B, Sousa M, Vieira R, Sousa N, Picó-Pérez M, et al. Stress influences the effect of obsessive-compulsive symptoms on emotion regulation. Front Psychiatry 2021;11:594511.
crossref
30. Marco JH, Fernandez-Felipe I, Fonseca S, Garcia-Palacios A, Baños R, Guillen V. Confirmatory factor analysis and psychometric properties of the Emotion Regulation Questionnaire in participants with personality disorders. Clin Psychol Psychother 2021;28:1598-1606.
crossref pmid
31. Lee M, Lee ES, Jun JY, Park S. The effect of early trauma on North Korean refugee youths’ mental health: moderating effect of emotional regulation strategies. Psychiatry Res 2020;287:112707.
crossref pmid
32. Oh DH, Lee S, Kim SH, Ryu V, Cho HS. Low working memory capacity in euthymic bipolar I disorder: no relation to reappraisal on emotion regulation. J Affect Disord 2019;252:174-181.
crossref pmid
33. Lee Y, Kim BN, Park MH, Park S. Familial, cognitive, and behavioral characteristics of adolescents with depression. J Korean Acad Child Adolesc Psychiatry 2017;28:168-173.
crossref
34. Kim SH, Cornwell B, Kim SE. Individual differences in emotion regulation and hemispheric metabolic asymmetry. Biol Psychol 2012;89:382-386.
crossref pmid
35. Kim SA, Kim H, Kim SH. Reappraisal modulates attentional bias to angry faces. Front Psychol 2016;7:1841.
crossref pmid pmc
36. Velotti P, Rogier G, Civilla C, Garofalo C, Serafini G, Amore M. Dysregulation of positive emotions across community, clinical and forensic samples using the Italian version of the difficulties in emotion regulation scale-positive (DERS-positive). J Forens Psychiatry Psychol 2020;31:555-570.
crossref
37. First MB, Williams JBW, Karg RS, Spitzer RL. User’s Guide for the SCID-5-CV Structured Clinical Interview for DSM-5® Disorders: Clinical Version. Arlington, VA: American Psychiatric Publishing, Inc; 2016.

38. Shon J. Individual Differences in Two Regulation Strategies: Cognitive Reappraiser vs. Emotion Suppressor. [dissertation]. Seoul: Seoul National University; 2005.

39. Lee M, Jang KS. Mediating effects of emotion regulation between socio-cognitive mindfulness and achievement emotions in nursing students. Healthcare (Basel) 2021;9:1238.
crossref pmid pmc
40. Beck AT, Steer RA, Brown GK. Beck Depression Inventory (BDI-II). San Antonio, TX: Psychological Corporation; 1996.

41. Lim SU, Lee EH, Hwang ST, Hong SH, Kim JH. Psychometric Properties of the Beck Depression Inventory-II in Korea. Poster presented at: Fall Conference of the Korean Clinical Psychology Association 2014;Korea.

42. Taylor S, Zvolensky MJ, Cox BJ, Deacon B, Heimberg RG, Ledley DR, et al. Robust dimensions of anxiety sensitivity: development and initial validation of the Anxiety Sensitivity Index-3. Psychol Assess 2007;19:176-188.
crossref pmid
43. Lim YJ, Kim JH. Korean anxiety sensitivity index-3: its factor structure, reliability, and validity in non-clinical samples. Psychiatry Investig 2012;9:45-53.
crossref pmid pmc
44. Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The posttraumatic stress disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J Trauma Stress 2015;28:489-498.
crossref pmid
45. Kim JW, Chung HG, Choi JH, So HS, Kang SH, Kim DS, et al. Psychometric properties of the Korean version of the PTSD Checklist-5 in elderly Korean veterans of the Vietnam war. Anxiety Mood 2017;13:123-131.

46. Joe KH, Chai SH, Park A, Lee HK, Shin IH, Min SH. Optimum cut-off score for screening of hazardous drinking using the Korean version of alcohol use disorder identification test (AUDIT-K). J Korean Acad Addict Psychiatry 2009;13:34-40.

47. Kline RB. Principles and Practice of Structural Equation Modeling. 4th ed. New York, NY: Guilford Publications; 2015.

48. Muthén LK, Muthén BO. Mplus User’s Guide. 7th ed. Los Angeles, CA: Muthén & Muthén; 1998.

49. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Modeling 1999;6:1-55.
crossref
50. Brown MW, Cudeck R. Alternative ways of assessing model fit. Sociol Methods Res 1992;21:230-258.
crossref
51. IMB Corp. IBM SPSS Statistics for Windows, Version 22.0. Armonk, NY: IBM Corp; 2013.

52. DeVellis RF. Scale Development Theory and Applications. 2nd ed. Los Angeles, LA: SAGE Publications; 2003.

53. An X, Yung YF. Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It. SAS Institute Inc. 2014;Available at: https://support.sas.com/resources/papers/proceedings14/SAS364-2014.pdf. Accessed Aug 1, 2021.

54. Preece DA, Becerra R, Hasking P, McEvoy PM, Boyes M, Sauer-Zavala S, et al. The emotion regulation questionnaire: psychometric proper ties and relations with affective symptoms in a United States general community sample. J Affect Disord 2021;284:27-30.
crossref pmid
55. Mohammed AR, Kosonogov V, Lyusin D. Expressive suppression versus cognitive reappraisal: effects on self-report and peripheral psychophysiology. Int J Psychophysiol 2021;167:30-37.
crossref pmid
56. Butler EA, Egloff B, Wilhelm FH, Smith NC, Erickson EA, Gross JJ. The social consequences of expressive suppression. Emotion 2003;3:48-67.
crossref pmid
57. Sheppes G. Emotion Regulation Choice: Theory and Findings. In: Gross JJ editors. Handbook of Emotion Regulation. 2nd ed. New York, NY: Guilford Publications; 2014. p.126-139.

58. Haga SM, Kraft P, Corby EK. Emotion regulation: antecedents and well-being outcomes of cognitive reappraisal and expressive suppression in cross-cultural samples. J Happiness Stud 2009;10:271-291.
crossref
59. Melka SE, Lancaster SL, Bryant AR, Rodriguez BF. Confirmatory factor and measurement invariance analyses of the emotion regulation questionnaire. J Clin Psychol 2011;67:1283-1293.
crossref pmid
60. Campbell-Sills L, Barlow DH, Brown TA, Hofmann SG. Acceptability and suppression of negative emotion in anxiety and mood disorders. Emotion 2006;6:587-595.
crossref pmid
61. Beblo T, Fernando S, Klocke S, Griepenstroh J, Aschenbrenner S, Driessen M. Increased suppression of negative and positive emotions in major depression. J Affect Disord 2012;141:474-479.
crossref pmid
62. D’Avanzato C, Joormann J, Siemer M, Gotlib IH. Emotion regulation in depression and anxiety: examining diagnostic specificity and stability of strategy use. Cognit Ther Res 2013;37:968-980.
crossref
63. Moore SA, Zoellner LA, Mollenholt N. Are expressive suppression and cognitive reappraisal associated with stress-related symptoms? Behav Res Ther 2008;46:993-1000.
crossref pmid pmc
64. Gul A, Khan K. Emotion regulation strategies can predict task-switching abilities in euthymic bipolar patients. Front Hum Neurosci 2014;8:847.
crossref pmid pmc
65. Shushakova A, Ohrmann P, Pedersen A. Exploring deficient emotion regulation in adult ADHD: electrophysiological evidence. Eur Arch Psychiatry Clin Neurosci 2018;268:359-371.
crossref pmid
66. Materna L, Wiesner CD, Shushakova A, Trieloff J, Weber N, Engell A, et al. Adult patients with ADHD differ from healthy controls in implicit, but not explicit, emotion regulation. J Psychiatry Neurosci 2019;44:340-349.
crossref pmid pmc
67. Boden MT, Gross JJ, Babson KA, Bonn-Miller MO. The interactive effects of emotional clarity and cognitive reappraisal on problematic cannabis use among medical cannabis users. Addict Behav 2013;38:1663-1668.
crossref pmid
68. Perez CR, Soto JA. Cognitive reappraisal in the context of oppression: implications for psychological functioning. Emotion 2011;11:675-680.
crossref pmid
69. Badour CL, Blonigen DM, Boden MT, Feldner MT, Bonn-Miller MO. A longitudinal test of the bi-directional relations between avoidance coping and PTSD severity during and after PTSD treatment. Behav Res Ther 2012;50:610-616.
crossref pmid pmc
70. Chung MC, Shakra M, AlQarni N, AlMazrouei M, Al Mazrouei S, Al Hashimi S. Posttraumatic stress among Syrian refugees: trauma exposure characteristics, trauma centrality, and emotional suppression. Psychiatry 2018;81:54-70.
crossref pmid
71. Short NA, Boffa JW, Clancy K, Schmidt NB. Effects of emotion regulation strategy use in response to stressors on PTSD symptoms: an ecological momentary assessment study. J Affect Disord 2018;230:77-83.
crossref pmid pmc
72. Joormann J, Gotlib IH. Emotion regulation in depression: relation to cognitive inhibition. Cogn Emot 2010;24:281-298.
crossref pmid pmc


ABOUT
AUTHOR INFORMATION
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
Editorial Office
#522, 27, Seochojungang-ro 24-gil, Seocho-gu, Seoul 06601, Korea
Tel: +82-2-717-0892    E-mail: psychiatryinvest@gmail.com                

Copyright © 2022 by Korean Neuropsychiatric Association.

Developed in M2PI

Close layer
prev next