According to a 2016 national epidemiologic study representing the psychiatric community in Korea, the 12-month prevalence of schizophrenia was 0.2% [1]. This prevalence rate of schizophrenia may be an underestimate, because people who were admitted to hospitals, nursing homes, and special social facilities and households with six or more non-family members were excluded from the survey.

In 2016, the prevalence of schizophrenia was reported to be as high as 0.6% based on the database of the Health Insurance Review and Assessment Service (HIRA) in Korea [2]. Since the HIRA’s database of insurance claims data represents the number of patients with schizophrenia currently being treated, it is also possible that this prevalence of schizophrenia is underestimated because patients not yet diagnosed with schizophrenia were not included.

In order to provide the appropriate care and management for people with schizophrenia in the community, it is necessary to accurately estimate a prevalence rate of schizophrenia that considers the magnitude of undiagnosed persons with schizophrenia.

Lengthy interview instruments exist for the assessment of psychotic disorders, but they often require accredited training to administer, and their length means they may not be appropriate for all situations [3]. Validated screening instruments provide a useful alternative to the full assessment tools for a disorder [4]. The Psychosis Screener (PS) was developed by researchers as the first stage of screening in the Australian National Survey of Mental Health and Well-Being (NSMHWB) Survey of Adults to find undiagnosed cases of psychosis in the community [5].

The aim of this study was to examine the validity of the Korean-translated version of the PS, compared to full diagnoses of psychotic disorders using the International Classification of Diseases, 10th edition (ICD-10) [6], in order to provide appropriate management for people with psychosis in the community.

Among the 126 psychiatric outpatients who were enrolled in a consecutive manner during the study period, the prevalence of psychosis was 15.1%. There were no significant differences in age or sex between the psychosis and the non-psychosis groups.

Of the 107 patients classified as non-psychosis, 6 (5.6%) were diagnosed with bipolar disorder, 55 (51.4%) with depression, 22 (22.9%) with anxiety or panic disorder, 9 (8.4%) with PTSD, 6 (5.6%) with insomnia, and 6 (5.6%) with dependence syndrome. Three patients (3.1%) were classified as having other conditions such as Alzheimer’s disease and alcohol dependence (Table 1).

The internal consistency of the PS in the current study was high (Cronbach’s α=0.81; 95% CI 0.65-0.91). Test-retest reliability was fair (r=0.69). At retest, the participants rated their psychotic symptoms as slightly more severe than at the initial test, but the difference was not statistically significant. Compared to the non-psychosis group, patients diagnosed with psychosis reported experiencing more delusions and hallucination. Among the psychosis cases, 63.2% reported experiencing hallucination, and the proportion that reported experiencing symptoms was 57.9% for both delusional mood and delusions of persecution (Figure 1).

The screener showed 78.9% sensitivity and 72.0% specificity when the optimal cut-off was 2, indicating that a score of 2 or more on the screener identified a likely case of psychosis. The area under the curve (AUC) for the PS was 0.78 (95% CI: 0.67-0.89). The positive predictive value was 33.3%, and the negative predictive value was 95.1% in an outpatient psychiatric clinic (Figure 2, Table 2).

Forty-three participants showed an inconsistency between the PS test results and the clinical diagnosis. Of these, 3 were false negative, and 40 were false positive. All three false negatives were female. Among the 40 false positives, 5 were diagnosed with bipolar disorder, 4 with PTSD, 24 with depression, 4 with anxiety or panic disorder, one with insomnia, and 2 with dependence syndrome. Most of the false positive patients were adults aged 19-64, and only 12.5% were aged 65 or older (Table 3).

This study was conducted to validate the Korean version of a short screening instrument designed for the detection of psychosis. The participants were enrolled at a psychiatric outpatient clinic in a consecutive manner during the study period to avoid selection bias. The prevalence of psychosis (F20-F29) in the current study was 15.1%. The proportion of schizophrenia patients among psychiatric outpatients in general hospitals is known to be around 5%. The clinics that participated in the current study were hospitals operated by the government with medical expenses for low-income families; therefore, the rate of schizophrenia in this outpatient clinic was relatively high.

The gold standard of psychosis was independently decided by two psychiatrists, one diagnosed after direct examination of the patient, and the other made a diagnosis based on a review of the patient’s medical chart. The final diagnoses obtained from the two methods were compared, and if there was any discrepancy, the two psychiatrists came to a consensus.

Since the PS test was independently administered and recorded by a clinical psychologist who did not know the patient’s diagnosis, the possibility of information bias might be minimal. The retest of the PS was conducted 4 weeks after the initial evaluation to prevent learning effects from taking the initial test from affecting the retest results.

The sensitivity and specificity of the PS test for the diagnosis of psychosis in an outpatient psychiatric clinic were 78.9% and 72.0%, respectively, and the reliability of the diagnosis of psychosis was 0.69. The PS is a simple screening tool measuring hallucination or delusions, which are the representative symptoms of psychosis. The PS is designed to allow mental health professionals to perform face-to-face interviews with patients and to additionally identify symptoms through nonverbal cues such as the facial expressions and gestures of patients during interviews. Nevertheless, three patients with schizophrenia responded that they had no symptoms of delusions or hallucinations, resulting in a false negative rate of 15.8%. All three people who classified as false negatives were women. Since PS is a screening tool that checks schizophrenia symptoms such as delusions and hallucinations in a relatively short time, it does not include probing questions that can sufficiently confirm the history of schizophrenia. Currently, there is a possibility that a false negative may appear in patients who have sufficiently improved or disappeared hallucinations or delusional symptoms at the time of interview. Among them, there is a symptom of schizophrenia, but in the case of one delusion without hallucinations or vice versa, it is possible that they did not satisfy the cut off 2 or more points. Lack of awareness of being ill is present in up to 40%-50% of those with untreated schizophrenia [7]. Especially, it has been reported that patients with schizophrenia frequently do not recognize the name of schizophrenia due to their stigma in Korea [8].

When applying this tool, there is a possibility of a false negative result due to potential patient denial of their psychotic symptoms, so even if the patient does not report it directly, it is recommended that the patient be interviewed by a mental health professional who can catch any delusional or hallucinatory characteristics of the patient and ask specific questions to confirm psychotic symptoms. If the interviewer has already established a rapport with the patient, the sensitivity of the screening test might be increased [9].

The false positive rate of the PS was 28% in the current study. Almost two-thirds of the false positives were patients with depression. The reason might be due to the fact that depression can be accompanied by psychotic features. Of the depression patients, 19% had a diagnosis of depression with psychotic features. Psychotic depression is a major depressive episode that is accompanied by psychotic symptoms [10]. It can be difficult to distinguish from schizoaffective disorder in screening tests that evaluate only the presence or absence of symptoms.

Of the false positives, 12.5% were patients with bipolar disorder. They are more likely to have hallucinations if they experience severe changes in mood. PTSD accounted for 10% of false positives. For individuals with PTSD, hearing voices was correlated with dissociative experiences. Although, if present, hallucinations and delusions in PTSD patients tend to be more paranoid and persecutory in nature compared with those observed in patients with primary psychotic disorders, which are more bizarre [11]. PS is a screening tool that evaluates only the presence or absence of hallucinations and delusions and does not include a differential diagnosis process. There were many false positives in the diagnosis that would require a differential diagnosis process by a psychiatrist.

The PS showed 78.9% sensitivity and 72.0% specificity. When this screening tool was applied to a group of prisoners with a prevalence of psychosis of 10% [12], the PPV was 24%. This means that 24 out of 100 positives resulting from the PS screening would be actual cases of psychosis. When this screening tool was applied to homeless people with a prevalence of psychosis of 20% [13], the PPV was 41%. This means that 41 out of 100 positives resulting from the PS screening would be actual cases of psychosis. When considering the PPV, effective screening for early examination of psychosis seems to be possible only in groups with high prevalence.

In psychosis screening, the favorable PPV recorded in a high-prevalence clinical population will fall sharply when the same screening tool is used in the low-prevalence context of the general population. Therefore, if someone wants to estimate the prevalence of psychosis in a community setting, screening could be assessed using a two-stage method. The PS might be used as an initial brief screen in which participants can report psychotic symptoms; all screen positives would then be administered the diagnostic interview for clinical assessment of psychosis. The second stage will identify false positives with a random selection of screen negatives that is essential in identifying false negatives. First-stage screening assessments are rarely perfectly sensitive. If the proportion of false positives is substantially large, the burden of administering a two-stage diagnostic interview will be increased [14].

The PS can also be applied to find those undiagnosed with severe mental illness with a broad definition of psychosis in the community setting. The definition of severe mental illness was a diagnosis of either schizophrenia, schizoaffective disorder, schizophrenia spectrum disorder (F20-F29); affective psychosis such as bipolar disorder (F31); and recurrent depressive disorder (F33) [15]. The PS was well able to discriminate between cases and non-cases using the clinician-diagnosed ICD-10 code as the standard. A score of 1 or more on the PS was the optimal score for severe mental illness. The sensitivity and specificity at this cut-off were 83.9% and 68.4%, respectively (Supplementary Figure 1 in the online-only Data Supplement).

We have to mentioned several limitations of the study. Firstly, the PS is a tool developed for the purpose of screening people with severe mental illness in the community, however it has been validated for patients in psychiatric hospitals because the prevalence of people with severe mental illness is very low in the community. Secondly, we did not include various psychopathological measures such as, the Brief Psychiatric Rating Scale (BPRS) and the Positive and Negative Syndrome Scale (PANSS) which are widely utilized in schizophrenia researches in the current study. Finally, the PS was not compared with other screening tools in terms of psychometric properties.

Estimation of severe mental illness prevalence was conducted in Australia using the two-phase survey strategies employed in both governmental and non-governmental mental facilities in 2010 using the PS. This involves the use of a brief and easy-to-administer screening instrument in the first phase. The second phase interviews are enriched with those more likely to meet diagnostic criteria. They estimated the prevalence of severe mental illness in the community by applying sampling weights derived from phase 1 to phase 2 data through a process known as Horvitz-Thompson inverse probability weighting [16].

In our sample, the prevalence of psychotic disorders was considerably higher than would be expected in a community setting. Hence, the sensitivity of the test in the community setting would be lower than that obtained in the current results, while the specificity would be higher.

This is the first study conducted on the Korean version of the PS. The instrument proved to be brief to administer (5-7 minutes) and have an ability to screen psychotic symptoms in an outpatient psychiatric clinic. The PS can be used as a screening tool to detect undiagnosed psychosis in a population with a high prevalence of psychosis, and it also might be used as a tool to accurately estimate the prevalence of people with psychosis or severe mental illness in the community.