Validity and Reliability of Cognitive Attentional Syndrome-1 Questionnaire
Article information
Abstract
Objective
This study aimed to evaluate the reliability and validity of the Turkish version of Cognitive Attentional Syndrome-1 (CAS-1) questionnaire.
Methods
221 participants were included in the study who do not meet any psychiatric diagnosis. Participants were applied SCID I and II and filled CAS-1 scale, Meta-Cognitions Questionnaire-30 (MCQ-30), Beck Depression Inventory (BDI), Beck Anxiety Inventory (BAI), Generalized Anxiety Disorder-7 (GAD-7) Scale, and Penn State Worry Questionnaire (PSWQ). Testing the reliability Cronbach’s alpha, item analysis and Item and total score correlation coefficients were applied. For testing structural validity, Confirmatory Factor Analysis was used, and for testing the content validity, the relationship between each item of CAS-1 and MCQ-30, BDI, BAI, GAD-7, PSWQ was examined.
Results
The correlation reliability coefficients were statistically significant except for using alcohol/drugs as a coping mechanism. Cronbach Alpha reliability coefficient of 16 items was 0.771 whereas, this ratio was 0.772 for the first eight items (CAS) and 0.685 for the last eight items (Metacognitive Beliefs) which showed that the internal consistency of CAS-1 was high. Structural and Content Validity of the scale was significant.
Conclusion
The Turkish version of the CAS-1 was a reliable and valid measure to evaluate CAS in a Turkish population.
INTRODUCTION
Metacognition is a specific term to describe a distinct category of thinking and cognition which control, modify and interpret the thinking processes. At the time of real or perceived crisis, metacognitions drive coping responses and may operate Cognitive Attentional Syndrome (CAS) [1,2]. The components of CAS are worry/rumination (perseverative thinking), threat monitoring (attentional strategies) and unhelpful coping strategies (thought suppression, avoidance, substance use). The perseverative thinking and threat monitoring may decrease cognitive capacity and thought suppression, avoidance, substance and alcohol use which may prevent proper learning experiences [2] which results in continuation of the psychopathology. CAS is operated via two distinct metacognitive beliefs about the components of the CAS, the former known as positive metacognitive beliefs and the latter as negative metacognitive beliefs [3]. Positive metacognitive beliefs include the usefulness or advantages of engaging to CAS, such as the usefulness of coping with worry, rumination, threat monitoring, gap filling, thought suppression, avoidance and substance use in response to the negative internal event. Negative metacognitive beliefs are related to the uncontrollability, meaning, importance, and dangerousness of thoughts and cognitive experiences [4]. The CAS and metacognitions are accepted as a transdiagnostic phenomenon that contributes to psychopathology with its different levels and aspects of its components.
To date, there has been no validated scale to evaluate the CAS in Turkish literature, yet there are validated scales to measure worry, rumination and metacognitions separately in the Turkish language. These scales are Penn State Worry Questionnaire [5] The Metacognitions Questionnaire-30 [6], Positive Beliefs About Rumination Scale [7], Negative Beliefs About Rumination Scale [8], Generalized Anxiety Disorder-7 (GAD-7) scale [9], Ruminative Thought Style Questionnaire [10]. Although these scales measure the severity of the worry, rumination, coping responses or metacognitions separately, none of them measure all components of the CAS and metacognitions with a single scale.
CAS-1 is a self-administered scale that exclusively assesses the frequency of the individual’s perseverative thinking styles and dysfunctional coping strategies and the degree of the metacognitive beliefs about these strategies [11]. CAS-1 is not a disorder-specific rating scale [4] which is one disadvantage of it. However, it is designed to be used when the diagnostic criteria do not meet, and the diagnosis is vague [4]. This research aims to provide validity and reliability of the CAS-1 scale in the Turkish language.
METHODS
Participants
Participants are invited from Haydarpaşa Numune Hospital medical board between May 2018 and August 2018. Inclusion criteria were being between 18 and 65 years old, being a volunteer to participate, not fitting with any psychiatric diagnosis according to SCID I or II. All participants had the cognitive competence for SCID I and II assessment and to fill the tests. Institutional Review Board approval date was 18 April 2018, protocol number 65. Informed consents were obtained from the volunteers who agreed to participate in the study and the rights of the participants have been protected according to the Declaration of Helsinki. Total 253 individuals accepted to participate in the study. Thirty-two participants were excluded from the study due to SCID I and/or SCID II diagnosis. As a result; 221 participants were included in the study (Approval number: 65).
Procedure
Approval for the Turkish adaptation of the scale was obtained from the developers of the original scale. A psychiatrist specialized in metacognitive theory and therapy have translated the scale into the Turkish language. Afterward, another psychiatrist who was blind to the research and procedure translated the Turkish text back to English. The latter English version was translated into Turkish again for the second time by two psychiatrists with professional experience and knowledge in the English language. Upon completion of translations, the research team drafted the final text. Lastly, a pilot test with 20 volunteers were conducted to control possible error and potential problems with the comprehension. No further revisions were required upon completion of the pilot test since none of the participants reported any negative feedback. Following the pilot test, individuals were invited to participate in the study.
First, individuals were informed about the procedure and invited to participate in the study. Each participant was provided informed consent about the procedure. Latter SCID-1 and SCID-2 were applied to all participants. The volunteers with a psychiatric diagnosis and a personality disorder, according to SCID I and II were excluded from the study and invited to psychiatric inpatient clinic for treatment. Subsequently, sociodemographic data form, Cognitive Attentional Syndrome-1 scale, Meta-Cognitions Questionnaire-30, Beck Depression Inventory, Beck Anxiety Inventory, GAD-7 Scale, and Penn State Worry Questionnaire were filled by the participants.
Data collection tools
Socio-demographic form
This form was developed to collect information about demographics including age, sex, marital status, level of education, psychiatric and medical history, and alcohol and substance use.
Cognitive-attentional syndrome-1 scale
CAS-1 scale was developed by Wells to assess the activation of the cognitive-attentional syndrome [4]. The scale consists of sixteen items, and it is the only known tool developed to evaluate all elements of CAS simultaneously. It assesses the proportion of time engaged in worry/rumination, threat monitoring, and coping behaviors and levels of positive and negative metacognitive beliefs. The first two items assess the level of perseverative thinking style as worry and rumination and threat monitoring in the last week on 9-point Likert-style rating scales ranging from 0 to 8. The next six items assess the frequency of coping strategies used to deal with negative feelings or thoughts on 9-point Likert-style rating scales ranging from 0 to 8. The next eight items assess the level of metacognitive beliefs of individuals about their cognitions and the CAS. These eight items are evaluated with Likert’s type rating between 0–100. In order to calculate the total score, the first eight items’ scores were converted from 0 to 8 Likert-style rating to 0 and 100 Likert-type rating similar to the last eight items. All 16 items were calculated between 0-100 Likert rating to form a total score. A high CAS-1 score indicates an increased level of CAS activation. Internal consistency of CAS-1 was significantly high (Cronbach’s alfa=0.86) [11].
Meta-Cognitions Questionnaire-30 (MCQ-30)
MCQ-30 was developed by Wells and Cartwright-Hatton [6] to assess individual differences in metacognitive beliefs, judgments, and monitoring tendencies. The scale has 30 items and is answered on a 4- point Likert-type rating scale with the options “(1) strongly disagree” and “(4) strongly agree.” Scoring of the scale is ranged between 30 to 120. The scale consists of five subscales which are positive beliefs about worry, negative beliefs about thoughts concerning uncontrollability and danger, cognitive confidence, beliefs about the need to control thoughts and cognitive self-consciousness. The increase in the total score of the scale indicates increased pathological metacognitive activity. The Turkish validity and reliability of the scale were made in 2008 by Tosun and Irak [12].
Beck Depression Inventory (BDI)
BDI was used to determine the level of depressive symptoms. The scale was developed to measure the physical, emotional, cognitive, and motivational symptoms of depression. It consists of 21 items and is answered on a 4 Likert type between 0–3 [13]. The validity and reliability of the Turkish version were made by Hisli [14] in 1988.
Beck Anxiety Inventory (BAI)
BAI is a self-reported scale that determines the frequency of anxiety symptoms experienced. The high scores indicate the severity of the anxiety experienced by the individual. It consists of 21 items and is a measure of 4 Likert type between 0–3 [15]. Ulusoy [16] made the validity and reliability of the Turkish version in 1993.
Generalized Anxiety Disorder-7 Scale
GAD-7 scale is a self-report measure developed by Spitzer et al. [9] to assess generalized anxiety disorder. It consists of 7 items and is an answered on a 4 Likert type between 0–3. The validity and reliability of the Turkish version were made by Konkan et al. [17] in 2013.
Penn State Worry Questionnaire (PSWQ)
PSWQ was developed in 1990 by Meyer et al. [5] in order to identify extreme, persistent and uncontrollable levels of pathological anxiety. It consists of 16 items and is answered on a 5 Likert type between 1–5. The validity and reliability of the Turkish version were made by Boysan et al. [18] in 2008.
Statistical analysis
Numerical variables were presented as mean and categorical variables as a percentage in the examination of the sociodemographic and clinical characteristics of the participants. Cronbach alpha internal consistency analyzes both the total and subscales of the questionnaire were performed for the reliability analyzes. Also, item-total score and subscales-total score correlation coefficients were assessed for the reliability parameters. To test the validity of the scale, Pearson’s correlation analysis was used via assessing the relationship between CAS-1 and MCQ-30, BDI, BAI, GAD-7, PSWQ. Factor analysis was performed to assess structural validity, but Varimax rotation was not performed because a single factor structure emerged. p≤0.05 was considered statistically significant. All data were analyzed using SPSS-20.0 for Windows (IBM Corp., Armonk, NY, USA).
RESULTS
The mean age of the participants was 27.44±2.86 years. 76.5% of the volunteers were female, and 23.5% were male. 3.7% primary school, 56.5% high school and 39.8% were university graduates. 72 participants were married, 149 participants were unmarried. Socio-demographic data is presented in Table 1.
Participants’ socio demographic characteristics
Internal consistency
Item and total score correlation coefficients for the items were evaluated. The correlation reliability coefficients were statistically significant except for item 3.5 which is used alcohol/drugs as a coping mechanism. The relationship between the item and the total score was found to be positive (p≤0.01) except for 3.5 item. Item and total score correlation coefficients are presented in Table 2.
Item-total score correlation coefficients and Cronbach’s alpha value and factor loads
Cronbach Alpha correlation analysis was used to determine internal consistency. According to the Cronbach Alpha correlation analysis result, Cronbach Alpha reliability coefficient of 16 items was 0.771. Also, this ratio was 0.772 for the first eight items (Cognitive Attentional Syndrome) and 0.685 for the last eight items (Metacognitive Beliefs). These results showed that the internal consistency of CAS-1 was high. Cronbach alpha values are presented in Table 2 if the items are deleted.
A Pearson correlation analysis was used to examine the relationships among the 16 items forming CAS-1. Selective relations have been obtained among the items as seen in Table 3. Significant correlation coefficients ranged from 0.139 to 0.739, and these coefficients were significant at 0.05 level. The analysis results are presented in Table 3.
The relations between the items of the Turkish version of CAS-1
Structural validity
The Confirmatory Factor Analysis was used in the factor analysis which evaluated the structural validity of CAS-1. The Kaiser-Meyer-Olkin value which is a measure of compliance of the scale items to the principal component analysis was found to be 0.713. The Bartlett value was found to be c2=995.971; p=0.001. In the correlation matrix, no item with a correlation coefficient greater than 0.9 was found. The rotation was not performed because the single factor structure was known. Factor loads were between 0,315 and 0,738, excluding item 3.5 (factor value is 0.061). Factor loads are presented in Table 2.
Content validity
In order to assess the content validity of CAS-1, the relationship between each item of CAS-1 and MCQ-30, BDI, BAI, GAD-7, PSWQ was examined. When the results of correlation are examined; item 1 and 2 seem to have a strong positive correlation with BDI, BAI, GAD-7, and PSWQ. When other items are examined, statistically significant correlations are seen. As a result, there is a positive correlation between MCQ-30, BDI, BAI, GAD-7, and PSWQ when the correlations of the CAS-1 total score are examined (0.468, 0.250, 0.249, 0.353, and 0.254 respectively) in Table 4. The current results show the validity of the CAS-1 Turkish version.
Pearson’s correlation analysis was used to assess the relationship between CAS-1 and MCQ-30, BDI, BAI, GAD-7, PSWQ
DISCUSSION
The study aimed to adapt Cognitive Attentional Syndrome Scale 1 (CAS-1) to Turkish language and to demonstrate validity and reliability of this adaptation. Due to the lack of reliability and validity studies of CAS-1 scale in other languages, we used other anxiety, worry, and metacognitive scales the (MCQ-30, BDI, BAI, GAD-7, and PSWQ) for comparative reasons in order to measure the psychometric properties of CAS-1.
Cronbach alpha values of the MCQ-30, PSWQ, BAI, BDI, GAD-7 are 0.86, 0.81, 0.93, 0.80, 0.852, respectively. Total items correlation coefficients ranging between 0.46 and 0.72 for the PSWQ and 0.395 and 0.72 for the GAD-7.
In our study, Cronbach alpha internal consistency analyzes of both the sum and subscales of the scales were performed for the reliability analysis. The internal consistency analyzes were found to have acceptable reliability values for 16 items of CAS-1 (Cronbach alpha=0.771>0.70) when assessed by the criteria proposed by Nunnally [19]. The correlation reliability coefficients were statistically significant except for the item 3.5 (used alcohol/drugs as coping) and the relationship between the item and the total score was found to be significant (p≤0.01). Lack of correlation between alcohol and drug use (as one of the maladaptive metacognitive strategies) as CAS activation in our study might be explained through using alcohol and drugs to cope with negative feelings, thoughts and beliefs might not be typical in Turkey due to cultural and religious factors. Additionally, the majority of our study population consisted of female adults (%76.5) and alcohol and drug use in females might not be frequent as in males in Turkey [20].
Moreover, coping strategies relating with negative feelings, thoughts and beliefs influenced by cultural values and genders [21-23]. According to item-total correlation analysis, interpretation of the items which were 0.30 or higher significantly distinguish the measured parameters [24]. The item-total test correlation coefficients were higher than the 0.30 cut-off point for all items, and the majority of the items had correlation coefficients higher than 0.50 which indicated a very high internal consistency. Cronbach Alpha reliability coefficient for the first eight items was 0.772 which measure the level of the CAS and 0.685 for the last eight items which measure Metacognitive Beliefs about the CAS. The results showed that the internal consistency of CAS-1 was high. The Kaiser-Meyer-Olkin value was 0.713; the Bartlett value was c2=995.971; p=0.001 and factor loads were between 0,315 and 0,738, excluding item 3.5 (factor value is 0.061) which showed the structural validity of the scale. A positive correlation between CAS-1 total score and MCQ-30, BDI, BAI, GAD-7, PSWQ were found (0.468, 0.250, 0.249, 0.353 and 0.254 respectively) which constituted evidence for the content validity of the Turkish form of CAS-1 scale. These results presented showed the validity of the CAS-1 Turkish version.
One of the shortcomings of our study was that we did not include any psychiatric diagnosis, this might be the reason why the correlation levels between CAS-1 and BDI and BAI were relatively lower. However, certain disorder-specific scales can be used in specific disorders but CAS-1 is intended for use when diagnosis is uncertain, or the patient does not meet the specific diagnostic criteria of a disorder [4]. This is the reason why we included in our study individual who does not meet the full diagnostic criteria of any specific disorder. Additionally, CAS-1 assess all the dimensions of the CAS and metacognitions, and it can be preferred over to use multiple scales to assess each dimension separately. Higher CAS-1 scores mean the CAS is severely activated, in other words, individuals who had higher levels of dysfunctional metacognitive beliefs and engaged in maladaptive metacognitive strategies such as rumination, worry, attention focusing to the threat, avoidance and thought suppression get higher scores from CAS-1.
To sum up, CAS-1 is a valid and reliable measure that exclusively assess the frequency of perseverative thinking styles and coping strategies and the degree of metacognitive beliefs about these strategies. This scale has acceptable psychometric values, a modest number of questions thus enabling the efficient and effective screening of worry, rumination, coping responses that backfire and metacognitions.
All in all, the CAS-1 is a brief, efficient and effective scale which encompasses all aspects of the cognitive attentional syndrome, even can be used when the diagnosis is uncertain. It is also a valuable contribution to short assessment tools when the time is limited, or the diagnostic criteria are not fulfilled. It is essential for the scale not to be considered as a stand-alone diagnostic measure used to diagnose any dysfunctions. This scale measures dysfunctional responses and coping strategies and metacognitive beliefs about these strategies.
Acknowledgements
We would like to thank young participants who took time to attend in this research.
Notes
The authors have no potential conflicts of interest to disclose.
Author Contributions
Conceptualization: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Mehmet Zihni Sungur. Data curation: Anıl Gündüz, Sencan Sertçelik, Betül Hacer Engin, Aysel İşler, Hatice Gönül, Arif Çipil. Formal analysis: İbrahim Gündoğmuş, Alişan Burak Yaşar, Arif Çipil. Investigation: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Mehmet Zihni Sungur, Betül Hacer Engin, Arif Çipil. Methodology: Anıl Gündüz, İbrahim Gündoğmuş, Alişan Burak Yaşar, Mehmet Zihni Sungur, Hatice Gönül. Project administration: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Alişan Burak Yaşar, Betül Hacer Engin, Aysel İşler, Hatice Gönül, Arif Çipil. Resources: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Alişan Burak Yaşar,Betül Hacer Engin, Aysel İşler, Hatice Gönül, Arif Çipil. Software: İbrahim Gündoğmuş. Supervision: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Mehmet Zihni Sungur, Hatice Gönül. Validation: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Mehmet Zihni Sungur, Betül Hacer Engin, Arif Çipil, Hatice Gönül. Visualization: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Betül Hacer Engin, Arif Çipil, Hatice Gönül. Writing—original draft: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Betül Hacer Engin, Aysel İşler, Alişan Burak Yaşar, Hatice Gönül. Writing— review & editing: Anıl Gündüz, İbrahim Gündoğmuş, Sencan Sertçelik, Mehmet Zihni Sungur.