Psychiatry Investig Search


Psychiatry Investig > Volume 17(4); 2020 > Article
Kim, Kim, Han, Jhon, Kim, Lee, Kim, Na, Kang, Chung, and Yoon: Validation of the Korean Version of the 15-Item Community Assessment of Psychic Experiences in a College Population



To investigate the reliability and validity of the Korean version of the Community Assessment of Psychic Experiences-15 item positive scale (CAPE-15) in college students.


This study had two stages: initial screening with self-report questionnaires including the CAPE-15, and semi-structured interviews to investigate the instrument’s diagnostic validity. The initial screening involved 1,749 college students. The modified Korean version of Prodromal Questionnaire-16 item (mKPQ-16) was also administered. The criteria for ultra-high risk (UHR) of psychosis in the Comprehensive Assessment of At-Risk Mental States (CAARMS) were the gold standard for diagnosis.


Twelve of the interviewed subjects met the CAARMS criteria for UHR of psychosis. The area under the receiver operating characteristic curve was highest (0.936) for the CAPE-15 distress score (p<0.001). The use of 6 as the cutoff for the CAPE-15 distress score resulted in the best balance of sensitivity (91.7%) and specificity (85.2%), with a favorable positive predictive value of 32.4%. The coefficients of correlation between the CAPE-15 and mKPQ-16 were significant.


The Korean version of the CAPE-15 is a good instrument for screening for psychosis risk in collegiate settings. The validation of this scale could contribute to the early identification of psychosis in the Korean community.


The retrospective concept of “prodrome” was replaced with the terms “clinical high risk” (CHR) or “ultra-high risk” (UHR) of psychosis to emphasize the prospective nature of psychotic disorders [1]. The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) includes “attenuated psychosis syndrome” (APS) as a new diagnosis requiring further research [2]. This new diagnosis has been developed based on evidence that treatment and intervention in these UHR or CHR states can prevent or delay the onset of psychosis [3]. Timely screening for and identification of APS in young individuals in community and school settings are required [4].
To increase the efficiency of identifying individuals at risk of psychosis, a preferable strategy is a stepwise process, with initial screening via self-report and subsequent detailed assessment using a structured interview. Brief self-reported screening instruments for initial pre-diagnostic filtering of the risk of psychosis before entering into a structured in-depth diagnostic evaluation are needed [5,6]. In Korea, a few screening tools for UHR of psychosis have been validated, including the 16- item version of the Prodromal Questionnaire (PQ-16) [7,8], Prodromal Questionnaire-brief version (PQ-B) [9,10], and the Eppendorf Schizophrenia Inventory (ESI) [6,11]. However, calculation of the ESI score is somewhat complicated because there are many items and a mixture of validity items in the questionnaire. The positive predictive value (PPV) of the original PQ-16 was found to be low (13.8%). Compared with the number of items assessing perceptual abnormalities, the PQ-16 has relatively few items assessing thought disorders and cognitive disturbances. Therefore, to increase the PPV of the PQ-16, we modified Korean version of the PQ-16 (mKPQ-16) by adding three ESI items (ideas of reference, cognitive impairment, and persecutory ideas) [7].
The Community Assessment of Psychic Experiences-Positive 15-items scale (CAPE-15) is another brief, valid screening tool used to identify people at high risk of psychosis in the community [12]. This study examined the reliability and validity of the Korean version of the CAPE-15 in university students.


Subjects and procedures

This study had two stages: initial screening with self-report questionnaires, including the CAPE-15, and semi-structured interviews to investigate the instrument’s diagnostic validity. The semi-structured interviews were conducted by phone with subjects who gave informed consent by trained mental health professionals (clinical psychologists, nurses, and social workers). In the second evaluation, the subjects were assessed with the Comprehensive Assessment of At-Risk Mental States (CAARMS) [13] to investigate whether they truly had UHR symptoms. When investigators strongly suspected that subjects did not meet the CAARMS UHR criteria, they made final diagnoses of no UHR of psychosis. Subjects whose diagnoses were not confirmed by phone interviews were then interviewed face-to-face by experienced psychiatrists. Final diagnoses were reviewed at a research team meeting with experienced psychiatrists.
All study subjects were college students aged between 18 and 30 years. In the classroom setting, professors and research associates provided an explanation of the purpose and procedure of this study to the prospective student participants (n=3,000), 2,010 (67.0%) of whom voluntarily completed the anonymous self-screening questions. Of these, 261 students older than 30 years of age were excluded and 1,749 students were the final subjects for the initial analysis. Of the initial study population, 1,225 (70.0%) consented to potentially being contacted a second time and thus provided their phone number. In line with the procedure of our previous validation studies [7], individuals with CAPE-15 scores ≥4 (n=224, 18.3%) and with depressive symptoms regardless of CAPE-15 score (n=76, 6.2%) were selected for semi-structured interviews. Of the subjects for whom the next investigation was indicated (n=300), 191 (63.7%) participated in a semi-structured interview. From these procedure, 20 participants were invited for face-to-face interviews, and 14 subjects underwent the face-to-face interviews. Seven more college students who visited an early intervention center for the evaluation of psychosis-like symptoms during the study period were also included in the final study population for diagnosis validation after completing the same face-to-face interview.
The Institutional Review Board of Chonnam National University Hospital approved this study (CNUH-2018-228), and all participants provided informed consent. The initial prescreening was performed in September and October 2018 and the diagnostic interviews were conducted between October 2018 and March 2018.


The initial screening questionnaires included questions regarding demographic characteristics, in addition to the Patient Health Questionnaire (PHQ-9), mKPQ-16, and CAPE-15. The PQ-16 is a self-report screening questionnaire that assesses the presence of attenuated psychotic symptoms using dichotomous questions (true/false). The level of distress associated with each symptom is rated on a four-point scale ranging from “no distress” to “much distress.” The modified Korean version of the PQ-16 includes three additional item drawn from the ESI, in addition to the 16 items of the original instrument [7,8].
The original CAPE is a self-report screening questionnaire that comprised 42 items [14]. The CAPE-15 is a shorter subscale that has been widely used as a screening tool to identify people at UHR in a clinical setting [15-17]. It assesses the presence of psychotic-like experiences with a three-factor structure: persecutory ideation (five items), bizarre experiences (seven items), and perceptual abnormalities (three items). It measures both the frequency of and distress associated with these experiences. Each item uses a 4-point Likert scale from 0, ‘never,’ through ‘sometimes’ and ‘often,’ to 3, ‘nearly always.’ For each endorsed item, distress was rated on a 4-point Likert scale from 0, ‘not distressed,’ through ‘a bit distressed’ and ‘quite distressed,’ to 3, ‘very distressed,’ which produced a total score of 0-45. Higher scores indicated a higher frequency of psychotic experiences and an increased level of distress due to these experiences. Three authors (SWK, YSK, and JSY) translated the CAPE-15 into Korean after obtaining permission from its original authors. Bilingual translators majoring in medicine back-translated the CAPE-15 into English, after which the instrument was finally edited. No significant discrepancies between the translated and original instruments were identified.
The CAARMS was used as the gold standard for UHR diagnosis [13]. The Korean version of the CAARMS, which was developed through a standard translation/back-translation procedure (followed by a final revision based on feedback from the original authors) and applied in various studies validating screening measures for UHR populations, was also used in this study [6,18]. The original CAARMS consists of seven subscales, but symptomatic criteria for psychosis risk are based exclusively on positive symptom items (disordered thought contents, perceptual abnormalities, and disorganized speech). In this study, four items from the positive symptom subscale were administered to subjects for the diagnosis of UHR of psychosis.
The Korean version of the nine-item Patient Health Questionnaire (PHQ-9), validated by Han et al. [19,20], was administered to measure the level of depressive symptoms.

Statistical analyses

Using data from the screening samples, the degree of correlation between the CAPE-15 and mKPQ-16 scores was assessed using Spearman’s correlation coefficient to examine convergent validity. Associations among these scales and PHQ-9 were investigated with Spearman’s correlation. The internal consistency was estimated with Cronbach’s alpha. Using data from the subjects who underwent structured evaluations, we conducted receiver operating characteristic (ROC) analyses for the diagnosis of UHR by the CAARMS interview. We determined the optimal cutoff values for the CAPE-15 frequency and distress scores based on the sensitivity, specificity, PPV, and negative predictive value (NPV). The accuracy of a test depends on how well it separates the group being tested into those with and without the disease in question, and is measured by the area under the curve (AUC). The validity of the mKPQ-16 item total score, which we previously proved to be the most valid measure, was also calculated for the study subjects to compare with the CAPE-15. The test-retest reliability of the CAPE-15 was estimated using the intraclass correlation coefficient (ICC) in the subjects who went through face-toface interviews. The data were analyzed using SPSS for Windows (ver. 21.0; IBM Corp., Armonk, NY, USA), and p-values< 0.05 were deemed statistically significant.


Diagnostic results

The mean age of the subjects who participated in the first screening (n=1,749) was 21.8±1.9 years. Female participants were more numerous than male participants (64.3%). For the screening dataset, Table 1 shows the Spearman’s coefficients of correlation between the mKPQ-16 and CAPE. Scores for the CAPE-15 frequency and distress scores were significantly associated with the mKPQ-16 item total and distress scores (all p-values<0.001). The PHQ-9 scores were significantly associated with the CAPE and mKPQ-16 scores. The correlation between frequency and distress scores of the CAPE-15 was strong (r=0.878, p<0.001).

Receiver operating characteristic analysis

The mean age of the subjects who underwent further investigations (n=198) was also 20.8±2.1 years, and females (74.3%) predominated. Among the subjects who were interviewed, 12 met the CAARMS criteria for UHR of psychosis. Two subjects met subthreshold frequency criteria and 10 subjects met subthreshold intensity criteria. To predict CAARMS-based diagnoses of UHR of psychosis, ROC curves were plotted (Figure 1). The AUCs were all significant for all measures, which were put into the model (all p-values<0.001). The AUC of the CAPE-15 distress score was the greatest (0.936) among the three measures.

Validity and reliability

Table 2 summarizes the diagnostic validity of the CAPE-15 based on the AUCs and optimal cutoff points for sensitivity and specificity. For the CAPE-15 distress score, a cutoff score of 6 resulted in the best balance of sensitivity (91.7%) and specificity (85.2%), with a favorable PPV of 32.4%. For the CAPE-15 frequency score, a cutoff score of 6 resulted in the greatest sensitivity (100%), despite a relatively low specificity (67.6%) and a PPV of 16.9%. A cutoff item total mKPQ-16 score of 7 showed favorable sensitivity (83.3%) and specificity (79.1%), and a PPV of 20.8%.
Cronbach’s alpha values for the CAPE-15 frequency and distress items were 0.867 and 0.835, respectively. The test-retest reliability was evaluated using data from subjects who underwent face-to-face evaluations, and the ICC for the CAPE-15 frequency and distress scores was 0.659 (p=0.010) and 0.859 (p<0.001), respectively.


This study validated the Korean version of the CAPE-15 as a screening measure for psychosis risk in a community setting. The Korean version of the CAPE-15 showed good internal consistency and diagnostic validity, successfully distinguishing subjects at UHR of psychosis from the general population. It showed favorable convergent validity with the mKPQ-16. In this study, the cutoff score of the mKPQ-16 for identifying psychosis risk was the same and its PPV was similar to those of our previous validation study for the mKPQ-16.7 This suggests that the process used in the current study was credible and showed the stability of the mKPQ-16 as a screening tool. The sensitivities and specificities of the PQ-16, PQ-B, and ESI were below 80% in previous studies [6,7,10], but those of the CAPE-15 in this study were 91.7% and 85.2%, respectively. The AUROC and PPV were higher for the CAPE-15 than for the mKPQ-16. It appears that the CAPE-15 is a more valid and useful screening tool compared with scales previously validated for use in Korean populations, and thus should facilitate early identification of psychosis in Koreans.
In this study, the PPV of the CAPE-15 distress score was improved (32.4%) compared with previous screening tools [6,7,10], but it was still relatively low for a screening tool with strong predictive power. Nevertheless, when a tool is used to screen for a serious, low-prevalence disease, such as psychosis, the chosen cutoff point should provide good sensitivity, even when the PPV is low or the false-positive rate is high [6,21]. However, clinicians should not use a screening tool for psychosis risk syndrome to give a final diagnosis, but simply for pre-testing the general population to identify those needing further clinical interviews, and should realize that screening the general population for psychosis risk using a self-report questionnaire could result in a high false-positive rate unless other factors are considered [21-23].
The validity of the distress score of the CAPE-15 was better than that of the frequency score of the CAPE-15. Functional decline and high subjective distress should be considered when an individual is diagnosed with UHR of psychosis [7,13]. Furthermore, given the problem of the high false-positive rate and the potential stigma in the non-help seeking population, distress level is important for detecting the true at-risk population [23-25].
In this study, the CAPE-15 and mKPQ-16 scores were significantly associated with the scores of the PHQ-9. This correlation between psychosis risk symptoms and depression was compatible with previous studies [17,26]. Comorbid depression is present in about 40% of those at UHR for psychosis [27,28]. Pathogenesis for comorbid depression and psychosis-like symptoms should be further investigated. Moreover, therapeutic strategy such as cognitive behavioral therapy for at-risk mental states should aim to manage both psychotic and depressive symptoms [29,30].
This study had several limitations. First, this population was mainly non-help seeking. In addition, the subjects in this study were young adults. Previous studies showed that children and adolescents were more likely to report symptoms indicative of UHR. Therefore, the CAPE-15 cutoff score identified in this study should be applied with caution in younger populations, and when used in the clinical setting [7,31]. Second, selection bias may have affected the results due to the loss of subjects from the study population. Finally, the number of individuals undergoing face-to-face clinical interviews was low. However, the results of the mKPQ-16 were similar to those in our previous study, suggesting that our current study procedure is reliable.
In conclusion, this study showed that the CAPE-15 is a good screening instrument for use in community and school settings. In particular, the better validity of the CAPE-15 suggests that it is a promising alternative to other brief self-report screening tools that are currently used to detect UHR. This validation of a questionnaire with a small number of items may make it feasible to screen large numbers of young adults in the community and shorten the duration of untreated psychosis through prompt early intervention.


This study was supported by a grant of the Korean Mental Health Technology R&D Project, Ministry of Health & Welfare, Republic of Korea (HM15C1140, HL19C0015) and a grant from the Basic Science Research Program through the National Research Foundation of Korea (NRF2017R1A2B4010830).

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Sung-Wan Kim, Young-Chul Chung, Jin-Sang Yoon. Data curation: Jae-Kyeong Kim, Ji Hun Han, Hyun Ju Na. Formal analysis: Sung-Wan Kim. Funding acquisition: Sung-Wan Kim, Young-Chul Chung. Investigation: Ji Hun Han, Min Jhon, Ju-Wan Kim. Methodology: SungWan Kim, Young-Shin Kang, Young-Chul Chung, Jin-Sang Yoon. Project administration: Jae-Kyeong Kim, Hyun Ju Na. Supervision: Ju-Yeon Lee, Jae-Min Kim, Young-Shin Kang, Young-Chul Chung. Writing—original draft: Sung-Wan Kim. Writing—review & editing: all authors.

Figure 1.
The receiver operating characteristic (ROC) curves of the CAPE-15 and mKPQ-16 with CAARMS criteria of ultra-high risk of psychosis as a gold standard. CAPE15: 15-item Community Assessment of Psychic Experiences, mKPQ-16: modified Korean version of the Prodromal Questionnaire-16.
Table 1.
Spearman’s coefficients of correlation among the CPAE-15, mKPQ-16, and PHQ-9
CAPE-15 frequency CAPE-15 distress PHQ-9
CAPE-15 frequency 0.878* 0.465*
CAPE-15 distress 0.878* 0.419*
mKPQ-16 frequency 0.564* 0.596* 0.493*
mKPQ-16 distress 0.518* 0.578* 0.504*

* all p-values<0.001.

CAPE-15: 15-item Community Assessment of Psychic Experiences, mKPQ-16: modified Korean version of the Prodromal Questionnaire-16 item, PHQ-9: nine-item Patient Health Questionnaire

Table 2.
Diagnostic validity of the CPAE-15 and mKPQ-16 based on the area under receiver operating characteristics curve (AUROC) and the optimal cutoff points for sensitivity and specificity
Scale Score AUROC 95% CI p-value Cutoff Sensitivity Specificity PPV NPV
CAPE-15 Frequency 0.872 0.803-0.940 <0.001 6 100% 67.6% 16.9% 100%
Distress 0.936 0.892-0.980 <0.001 6 91.7% 85.2% 32.4% 99.2%
mKPQ-16 Item total 0.839 0.724-0.953 <0.001 7 83.3% 79.1% 20.8% 98.6%

CAPE-15: 15-item Community Assessment of Psychic Experiences, mKPQ-16: modified Korean version of the Prodromal Questionnaire-16 item, PPV: positive predictive value, NPV: negative predictive value, CI: confidence interval


1. Yung AR, McGorry PD, McFariane CA, Jackson HJ, Patton GC, Rakkar A. Monitoring and care of young people at incipient risk of psychosis. Schizophr Bull 1996;22:283-303.
crossref pmid pdf
2. American-Psychiatric-Association. Diagnostic and Statistical Manual of Mental Disorder. 5th Ed. Arlington, VA: American Psychiatric Publishing; 2013.

3. Nelson B, Yuen HP, Wood SJ, Lin A, Spiliotacopoulos D, Bruxner A, et al. Long-term follow-up of a group at ultra high risk (“prodromal”) for psychosis: the PACE 400 study. JAMA Psychiatry 2013;70:793-802.
crossref pmid
4. Lee JY, Chung YC, Kim JM, Shin IS, Yoon JS, Kim SW. School counselors’ recognition of the ultra-high risk for psychosis. Psychiatry Investig 2018;15:320-324.
crossref pmid pmc pdf
5. Addington J, Stowkowy J, Weiser M. Screening tools for clinical high risk for psychosis. Early Interv Psychiatry 2015;9:345-356.
crossref pmid
6. Chung YC, Kang NI, Im YJ, Kim SW, Cho IH, Lee YM, et al. Validation of the Korean version of the Eppendorf Schizophrenia Inventory as a screening measure to detect adolescents at ultra-high risk for psychosis. Early Interv Psychiatry 2013;7:71-79.
crossref pmid
7. Kim SW, Chung YC, Kang YS, Kim JK, Jang JE, Jhon M, et al. Validation of the Korean version of the 16-Item Prodromal Questionnaire in a non-help-seeking college population. Psychiatry Investig 2018;15:111-117.
crossref pmid pdf
8. Ising HK, Veling W, Loewy RL, Rietveld MW, Rietdijk J, Dragt S, et al. The validity of the 16-item version of the Prodromal Questionnaire (PQ-16) to screen for ultra high risk of developing psychosis in the general help-seeking population. Schizophr Bull 2012;38:1288-1296.
crossref pmid pmc pdf
9. Loewy RL, Pearson R, Vinogradov S, Bearden CE, Cannon TD. Psychosis risk screening with the Prodromal Questionnaire--brief version (PQB). Schizophr Res 2011;129:42-46.
crossref pmid pmc
10. Jang YE, Lee TY, Hur JW, Kwon JS. Validation of the Korean version of the Prodromal Questionnaire-Brief Version in non-help-seeking individuals. Psychiatry Investig 2019;16:109-114.
crossref pmid pmc pdf
11. Mass R, Girndt K, Matouschek AK, Peter PM, Plitzko N, Andresen B, et al. Introducing the Eppendorf Schizophrenia Inventory (ESI) as a psychometric method for schizotypy research. Pers Individ Dif 2007;42:525-534.
12. Capra C, Kavanagh DJ, Hides L, Scott J. Brief screening for psychosislike experiences. Schizophr Res 2013;149:104-107.
crossref pmid
13. Fusar-Poli P, Cappucciati M, Rutigliano G, Lee TY, Beverly Q, et al. Towards a standard psychometric diagnostic interview for subjects at ultra high risk of psychosis: CAARMS versus SIPS. Psychiatry J 2016;2016:7146341
crossref pmid pmc pdf
14. Stefanis NC, Hanssen M, Smirnis NK, Avramopoulos DA, Evdokimidis IK, Stefanis CN, et al. Evidence that three dimensions of psychosis have a distribution in the general population. Psychol Med 2002;32:347-358.
crossref pmid
15. Bukenaite A, Stochl J, Mossaheb N, Schäfer MR, Klier CM, Becker J, et al. Usefulness of the CAPE-P15 for detecting people at ultra-high risk for psychosis: Psychometric properties and cut-off values. Schizophr Res 2017;189:69-74.
crossref pmid
16. Núñez D, Arias V, Vogel E, Gómez L. Internal structure of the Community Assessment of Psychic Experiences- Positive (CAPE-P15) scale: evidence for a general factor. Schizophr Res 2015;165:236-242.
crossref pmid
17. Perez J, Russo DA, Stochl J, Clarke J, Martin Z, Jassi C. Common mental disorder including psychotic experiences: trailblazing a new recovery pathway within the improving access to psychological therapies programme in England. Early Interv Psychiatry 2018;12:497-504.
crossref pmid
18. Chung YC, Jung HY, Kim SW, Lee SH, Shin SE, Shin YM, et al. What factors are related to delayed treatment in individuals at high risk for psychosis? Early Interv Psychiatry 2010;4:124-131.
crossref pmid
19. Han C, Jo SA, Kwak JH, Pae CU, Steffens D, Jo I, et al. Validation of the Patient Health Questionnaire-9 Korean version in the elderly population: the Ansan Geriatric study. Compr Psychiatry 2008;49:218-223.
crossref pmid
20. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient Health Questionnaire. JAMA 1999;282:1737-1744.
crossref pmid
21. O’Toole BI. Screening for low prevalence disorders. Aust N Z J Psychiatry 2000;34(Suppl):S39-S46.
crossref pmid
22. Owoso A, Ndetei DM, Mbwayo AW, Mutiso VN, Khasakhala LI, Mamah D. Validation of a modified version of the PRIME screen for psychosis-risk symptoms in a non-clinical Kenyan youth sample. Compr Psychiatry 2014;55:380-387.
crossref pmid
23. Loewy RL, Johnson JK, Cannon TD. Self-report of attenuated psychotic experiences in a college population. Schizophr Res 2007;93:144-151.
crossref pmid pmc
24. Post SG. Preventing schizophrenia and Alzheimer disease: comparative ethics. Schizophr Res 2001;51:103-108.
crossref pmid
25. Kim SW, Polari A, Melville F, Moller B, Kim JM, Amminger P, et al. Are current labeling terms suitable for people who are at risk of psychosis? Schizophr Res 2017;188:172-177.
crossref pmid
26. Moritz S, Göritz AS, McLean B, Westermann S, Brodbeck J. Do depressive symptoms predict paranoia or vice versa? J Behav Ther Exp Psychiatry 2017;56:113-121.
crossref pmid
27. Kline ER, Seidman LJ, Cornblatt BA, Woodberry KA, Bryant C, Bearden CE, et al. Depression and clinical high-risk states: Baseline presentation of depressed vs. non-depressed participants in the NAPLS-2 cohort. Schizophr Res 2018;192:357-363.
crossref pmid
28. Fusar-Poli P, Nelson B, Valmaggia L, Yung AR, McGuire PK. Comorbid depressive and anxiety disorders in 509 individuals with an at-risk mental state: impact on psychopathology and transition to psychosis. Schizophr Bull 2014;40:120-131.
crossref pmid pdf
29. Morrison AP, French P, Walford L, Lewis SW, Kilcommons A, Green J, et al. Cognitive therapy for the prevention of psychosis in people at ultra-high risk: randomised controlled trial. Br J Psychiatry 2004;185:291-297.
crossref pmid
30. Kim SW, Jang JE, Lee JY, Lee GY, Yu HY, Park C, et al. Effects of group cognitive-behavioral therapy in young patients in the early stage of psychosis. Psychiatry Investig 2017;14:609-617.
crossref pmid pmc
31. Schimmelmann BG, Michel C, Martz-Irngartinger A, Linder C, Schultze-Lutter F. Age matters in the prevalence and clinical significance of ultra-high-risk for psychosis symptoms and criteria in the general population: Findings from the BEAR and BEARS-kid studies. World Psychiatry 2015;14:189-197.
crossref pmid pmc


Browse all articles >

Editorial Office
#522, 27, Seochojungang-ro 24-gil, Seocho-gu, Seoul 06601, Korea
Tel: +82-2-717-0892    E-mail:                

Copyright © 2024 by Korean Neuropsychiatric Association.

Developed in M2PI

Close layer
prev next