INTRODUCTION
Community-based studies over the past few decades have shown that a high proportion of the nonclinical populations report psychotic-like experiences (PLE) [1-6]. It has been suggested that individuals with PLE share demographic, functional, and pathophysiological characteristics with clinical populations and that they have an increased risk of transitioning to clinical conditions in the future [4,7-9]. With such increasing evidence, the continuum hypothesis of psychosis has been proposed, which states that the psychotic symptoms exist within a wide spectrum from subclinical psychotic experiences to a full diagnosis of psychosis [10-12], with varying severity and duration [5,8,13]. Accordingly, clinical attention to nonclinical groups with PLE has grown as investigations into these populations may provide a better understanding of the etiology of psychosis and other psychiatric disorders. Along with this trend, several terms indicating nonclinical populations with PLE have emerged. For instance, “Ultra-high risk (UHR)” for psychosis refers to people who are at high risk of developing psychosis although the severity of their current symptoms is not up to clinical diagnostic standards for psychotic disorders [14,15]. The Diagnostic and Statistical Manual of Mental Disorders, 5th edition also included “attenuated psychosis syndrome,” as a condition requiring further study as a means of minimizing duration untreated psychosis and promoting better outcomes in treating psychotic disorders [16]. Besides, recent studies have revealed that early intervention in the high-risk of psychosis may prevent and/or delay the onset of psychosis [17-19]. Such progression has emphasized the importance of early identification of high-risk groups in community samples, which can lead to early intervention before the transition to clinical psychosis, improving the prognosis of patients with psychotic disorders.
The process of identifying nonclinical populations with PLE requires thorough clinical interviews conducted by trained professionals. Comprehensive Assessment of At-Risk Mental States is one of representative assessment tools currently used in research settings [20]. However, it is time-consuming and impractical considering that most nonclinical populations with PLE do not visit hospitals, resulting in blind spots where high-risk groups cannot receive a proper intervention. It would be useful to develop improved screening tools that can easily identify the presence of psychotic symptoms and track the course of psychosis in community samples. Previous studies have revealed that self-reported questionnaires are valid in detecting UHR individuals and assessing psychotic symptoms [21]. By employing pre-diagnostic filtering via self-report, results can be used as references for further professional steps and can overcome limitations in clinical settings (i.e., lack of trained staff and time constraints), increasing the efficiency of identification.
Several screening tools for psychotic symptoms have been developed, among which Community Assessment of Psychic Experience-42 (CAPE-42) has been widely used in clinical and research environments [22,23]. CAPE-42 is a self-reported assessment tool developed to measure the frequency and distress of a lifetime of psychic experiences in the general population. Containing three dimensions (positive, negative, and depressive symptoms), it evaluates the overall psychotic experiences usually observed in schizophrenia. Along with its proven psychometric properties [24-27], it has played a key role as an essential assessment tool for research examining the continuum hypothesis of psychosis [8,28]. However, the validity and reliability of the Korean version of CAPE-42 (K-CAPE-42) have not yet been evaluated, though CAPE-15, comprising only positive symptoms, has been translated into Korean and validated [29]. Symptoms frequently observed in the prodromal stage include depression, increased anxiety, social isolation, and lack of motivation [30-32]. Given the heterogeneous symptoms in psychotic disorders, the simplified version may have limitations in assessing overall aspects of psychotic experiences for early detection of psychotic disorders.
Accordingly, the K-CAPE-42 seems to have the great potential to be effectively used in research and medical settings if validated as an assessment tool for individuals with PLE. Thus, we aim to evaluate the reliability and validity of the K-CAPE-42, examining whether it can be used as a measure of the degree of psychotic symptoms in the general population.
METHODS
This study was approved by the Institutional Review Board of Seoul National University Bundang Hospital (IRB no. B-2011-648-306), and we obtained informed consent from all participants. The present study was done as a part of an ongoing project, “Investigating the Mechanisms underlying Psychosis Associated with Childhood Trauma,” which aims to investigate the effect of childhood trauma on behavior, brain, and mental state.
Participants
The inclusion criteria of this study were: 1) participants who are aged 18 to 40 years, 2) those who are fluent in Korean to understand the instructions , and 3) those who have no current or lifetime history of psychotic-related diseases and psychiatric medications. Among 1,467 registered participants, 17 were excluded due to incomplete responses and/or being found to be younger or older than the age criterion.
Assessments
K-CAPE-42
In the same way as the original version, K-CAPE-42 involves 42 items, consisting of three dimensions: positive, negative, and depressive symptoms (20, 14, and 8 items, respectively). Each item contains two 4-point Likert scales. That is, participants’ lifetime prevalence of psychic experiences was first rated as “never,” “sometimes,” “often,” and “nearly always” (from 1 to 4). Next, they further indicated the degree of subjective distress as “not distressed,” “a bit distressed,” “quite distressed,” and “very distressed” (from 1 to 4).
Prior to the translation process of K-CAPE-42, we first obtained permission from the original author [1]. A clinical neuropsychologist, who is fluent in both English and Korean, translated the English version into Korean. After that, two professional psychiatrists (KET and KSY) participated in the back-translation process. Lastly, they evaluated whether translated words and phrases convey the same meaning and/or implications as the original tool despite cultural differences. As there were no significant differences between the original and the translated version, the final Korean version of CAPE with 42 items was developed (Supplementary Material in the online-only Data Supplement).
Scales used for the validation process
To evaluate the convergent and discriminant validity of the K-CAPE’s three symptom dimensions (positive, negative, and depressive), we employed several psychotic symptom-related scales. For convergent validity, each three CAPE dimensions and their corresponding tools are supposed to measure the same construct and be related to each other. Conversely, for discriminant validity, two measures should be measuring different constructs and unrelated. Thus, we assumed that two related scales would be more significantly correlated than other unrelated scales. For example, scales concerning positive symptoms would have higher correlation coefficients with CAPE-positive (CAPE-pos) dimension than with CAPE-negative (CAPE-neg) and CAPE-depressive (CAPE-dep) dimension scores. In the process of selecting scales, we considered whether it is widely used, validated, easily accessible, and measuring the same content.
The Dissociative Experiences Scale (DES-II) is a self-evaluated instrument for screening dissociative symptoms such as derealization/depersonalization, absorption, and amnesia [33]. A validation study of the Korean version of DES has been performed and proven to be valid to assess dissociative symptoms in clinical and nonclinical samples [34]. Of the three subscales, we used the depersonalization/derealization (DES-II-DD) subdomain for analysis, expecting a high correlation with the CAPE-pos dimension.
The Paranoia Scale (PS), consisting of 20 items, is a self-report screening instrument measuring the frequency of subclinical levels of paranoia, including ideas of persecution and reference [35]. The Paranoia Scale translated into Korean (K-PS) has been validated [36].
The Oxford-Liverpool Inventory of Feelings and Experiences (O-LIFE) is a 20-item self-report scale that measures psychosis-proneness covering four dimensions: “unusual experiences (e.g., magical thinking and perceptual disturbance),” “cognitive disorganization (e.g., poor attention and decision-making),” “introvertive anhedonia (e.g., reduced feelings of social and physical pleasure),” and “impulsive nonconformity (e.g., impulsive, anti-social mood and behaviors).” In this study, the dimension of introvertive anhedonia (O-LIFE-IA) and unusual experiences (O-LIFE-UE) were used for analysis to compare with positive and negative symptoms, respectively. The validity and reliability of the Korean version of these screening tools have not yet been investigated, and the validation process is in progress.
Statistical analysis
All statistical analyses were conducted using R (version 4.1.1; R Foundation for Statistical Computing, Vienna, Austria). For factor analysis, the R package lavaan was utilized (version 0.5-17) [39]. The validation process is illustrated in Figure 1.
Internal reliability
First, to verify the reliability of K-CAPE-42, assessing how well scale items are correlated with one another and measuring the same construct, we calculated Cronbach’s alpha coefficient for frequency and distress scores of the entire 42 items and those of three subfactor frequency scores (positive, negative, and depressive). Alpha values greater than 0.70 indicates an acceptable level of internal consistency for further analysis.
Confirmatory factor analysis
CAPE’s positive scale was initially developed in one dimension, but subsequent studies have proposed several multidimensional structures. Representatively, there are three-subfactors structure that distinguishes “strange experiences,” “hallucinations,” and “delusional ideations” and a five-subfactors structure that further decomposes delusional ideation into subdimensional “paranoia,” “grandiosity,” and “magical thinking” (Table 1).
We performed confirmatory factor analysis (CFA) using maximum likelihood estimation in the whole sample to investigate whether the original model and the multidimensional model fit our data. In order to assess the model fit, the goodness-of-fit indices including root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), and Tucker-Lewis index (TLI) were calculated to determine whether to maintain or improve the the original model. CFI and TLI of higher than 0.90, and RMSEA of less than 0.08, and SRMR less than 0.06 are considered to indicate that they are suitable for our data [42-44].
Exploratory factor analysis
In case of having unsatisfactory CFA model fits, we would need to further investigate to see if we could find a better factor solution. Before any further analysis, the entire sample was randomly divided into two subsamples. Using the first half random sample, exploratory factor analysis (EFA) was conducted to investigate better alternative factor solutions. We employed principal component analysis with a promax rotation as our method of factor extraction. In the next step, the follow-up CFA was employed for the factor models that were generated from EFA using data from the second random sample.
Convergent and discriminant validity
Lastly, the degree of correlation between the K-CAPE subscales (CAPE-pos, CAPE-neg, and CAPE-dep) and other corresponding scales (DES-II, PS, O-LIFE-UE, O-LIFE-IA, and PHQ-9) was analyzed using Spearman’s correlation coefficient to examine convergent and discriminant validity. This analysis included a total of 1,373 participants who completed all scales.
RESULTS
Demographic characteristics
A total of 1,450 participants aged from 18 to 42 years were included in the final analysis. The mean age of the participants was 26.84 (standard deviation=5.56) years, and there were more women than men in the total sample (n=1,011, 69.72%). The entire sample was randomly divided into two subsamples for EFA, as described in the “method” section. Since sample 1 and 2 did not differ in mean age, gender ratio, and weighted total frequency scores, the randomization was considered successful. Participants’ demographic characteristics are provided in Table 2.
Reliability
Cronbach’s alpha value of total frequency and distress scores were 0.921 and 0.924, respectively, which indicates excellent internal consistency [45]. In addition to the total scores, internal reliability of frequency scores for three subfactors (Table 3) was found to be good for further analysis.
Correlation analysis between frequency and distress was performed for the total score and each dimension. Pearson correlation was significant between total scores of frequency and distress (r=0.893, p<0.001), and was also significant in three hypothesized dimensions (r>0.800 and p<0.001 for all dimensions) (Figure 2).
Confirmatory factor analysis
Table 4 indicates the goodness of fit indices of the original three-dimensional and other hypothesized multidimensional model analyzed from the CFA. According to the CFA results, even though the model fit indices did not reach their respective optimal thresholds, they were within an acceptable range (RMSEA, SRMR, and RMR, but not CFI and TLI). Furthermore, multidimensional models showed relatively better quality compared to the original three-dimensional model, but it did not significantly improve the model fit.
Exploratory factor analysis and follow-up CFA
Since goodness of fit indices of the CFA did not reach the expected value, EFA was conducted to investigate another factor solution that optimally describes the data. Values of the Kaiser-Meyer-Olkin measure of sampling adequacy (MSA=0.936) and Bartlett’s sphericity test (χ2=23,239; p<0.001) were calculated, which suggested our data were factorable. As a result, EFA proposed multiple factor solutions with 8 eigenvalues >1 while the result of the scree plot indicated 3-5 factor solutions (Figure 3).
In a 3-factor solution, “negative-avolition” items were found to be loaded more consistently with depressive items than with the negative dimension (Table 5). The 4-factor solution separated positive symptoms into two factors: “positive-bizarre experiences” and “positive-delusional thoughts,” while, in addition to this, the 5-factor solution separated negative symptoms into two distinct factors: “negative-avolition (expressive)” and “negative-social (experiential).”
The three-factor model accounted for 38.9% of the total variance (depressive symptoms, 19.8%; positive symptoms, 11.8%; negative symptoms, 7.3%). The model fit statistics of the three-factor model showed an appropriate fit except for the CFI and TLI (Table 4). Based on the results of the follow-up CFA, we rejected four- and five- factor models because they did not significantly enhance the model fit.
Convergent and discriminant validity
We analyzed the correlations between three dimensions of K-CAPE (pos, neg, and dep) and their corresponding symptom scales. As presented in Table 6, K-CAPE-pos total scores had stronger positive correlations with total scores of DES-II-DD (r=0.595, p<0.001, df=1,371), O-LIFE-UE (r=0.639, p<0.001, df=1,371), and PS (r=0.643, p<0.001, df=1,371), while it has lower correlations with total scores on O-LIFE-IA (r=0.241, p<0.001, df=1,371) and PHQ-9 (r=0.455, p<0.001, df=1,371). On the other hand, K-CAPE-neg total score was highly correlated with O-LIFE-IA (r=0.566, p<0.001, df=1,371) the most, while the correlation with other scales was relatively lower. Lastly, K-CAPE-dep dimension scores had a higher correlation coefficient with PHQ-9 (r=0.726, p<0.001, df=1,371) than others.
DISCUSSION
This study aimed to evaluate the reliability and validity of the K-CAPE-42 for application as assessment tool in community samples. Our findings established favorable internal consistency and robust convergent (and divergent) validity with their relevant subscales. Factor analysis showed that fit indices of the three-dimensional original model derived from CFA were not optimal and that EFA proposed alternative factor solutions with different item loadings but did not significantly improve the model fit. Thus, our study provided evidence to support that the Korean version applying the original three-dimensional model is suitable.
According to CFA, it is found that the original three-dimensional model of K-CAPE-42 exhibited inconsistent fit indices; CFI and TLI values were below the cutoff (>0.95) while RMSEA, RMR, and SRMR were satisfactory (<0.05). However, this is consistent with previous findings that not all fit indices of the translated versions exceed the acceptable criteria [13,46,47]. Also, it was suggested that CFI is not a valid indicator because it has a negative bias when the correlations between items are generally low [24,42]. RMSEA is an absolute measure of fit that determines how well the hypothesized model fits the perfect fitting model, whereas CFI and TLI are incremental fit indices evaluating the hypothesized model compared to the baseline model in which no items covary [43]. It has been calculated that when RMSEA is less than 0.158, obtained CFI values would be too small [42]. In our analysis, as the RMSEA of the baseline model was 0.135 (less than 0.158), the CFI and TLI (and other incremental fit indices) may not be very informative [44]. Thus, it is reasonable to conclude based on RMSEA and SRMR rather than on CFI and TLI, which suggests the acceptability/adequacy of the original model.
On the other hand, EFA suggested alternative factor solutions in which item loadings differ from the original CAPE42. In factor solutions, several negative items (“avolition”) were consistently loaded with depressive items than negative items. However, this result is consistent with other previous validation studies on the translated version of CAPE-42 in which the depression factor included few negative items [25,46]. It should also be noted that the negative (especially “avolition” symptoms) and depressive domains are conceptually overlapping and have common phenomenological characteristics with negative symptoms [48-51]. Since K-CAPE-42 was conducted through self-report, and detailed information and phenomenology are required to distinguish it, it is reasonable to obtain different item loading results. For example, positive item 7 (“Feeling persecuted”) and item 22 (“People look at you oddly”) were repeatedly included with depressive items rather than positive items. Even though both items were originally designed to cover the “paranoia” domain of positive symptoms, they can also be regarded relating to “low self-esteem.” People with low self-esteem has a lack of confidence and feel worthless about themselves, which is deeply associated with depression and also acts as a risk factor for depression [52,53]. Despite the different item loadings, this seems acceptable when considering overlapping characteristics of psychotic disorder. However, these alternative factor solutions were not selected since the follow-up CFA on these solutions did not significantly improve the model fit. Thus, we concluded that the three-factor model was suitable, as in the originally proposed.
Lastly, the correlation coefficients between the K-CAPE-42 subscales and their corresponding measurements (e.g., CAPE-pos & DES-II vs. CAPE-pos & O-LIFE-IA) were the highest while the subscales were less correlated with scales of other dimensions, confirming convergent and divergent validity. As mentioned above, with respect to the EFA results that “negative-avolition” and “positive-paranoia” items were included in the depressive domain, the correlation between CAPE-neg and PHQ-9 (r=0.623, p<0.001) and between CAPE-pos and PHQ-9 (r=0.455, p<0.001) were relatively higher than those with other irrelevant scales (but lower than those with corresponding scales), which again indicates overlapping features of psychotic symptoms (i.e., depression with “negative” and “positive” subdomains).
Our study is the first validation report of the K-CAPE-42 which includes reliability, factor structure, and convergent-divergent validity. There have been multiple CAPE variants depending on the language of the translation, the validation process, the target population, and what items they included (CAPE-42 or CAPE-P15) [24-26,41,54-56]. Specifically, although there is one study that validated the K-CAPE-15, consisting only of positive items and involving only young adults for analysis [29], the present study can be further generalized as it includes a wider range of participants and all three subdomains. K-CAPE-15 conducted face-to-face clinical interviews to evaluate the diagnostic validity and determined the cutoff values for UHR detection. Yet, we excluded individuals who have been diagnosed with psychotic disorders from the study and did not conduct follow-up investigations; further research is needed to evaluate the usefulness of detecting whether participants developed a psychotic disorder. When choosing a scale, it would be advantageous to consider the pros and cons of these different versions of CAPE, depending on the purpose of the measurement. In addition, due to the self-reporting system and recruitment via online social media, there might be potential biases such as response bias (e.g., socialdesirability bias, over or underreported symptoms) and selection bias, which possibly influence the quality of data. Nevertheless, even with such limitations, K-CAPE-42 can be effectively used as a pre-screening tool for “at-risk” populations.
In conclusion, this study showed that the K-CAPE-42 is a valid and effective screening tool with sufficient psychometric properties to measure PLE in community settings. Although the K-CAPE-42 is not a diagnostic tool for UHR, it aims to pre-screen individuals at potential risk for psychotic disorders, which consequently facilitates early identification and intervention as well as psychosis research.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Supplement
Print
