Properties of Patient-Reported Outcome Measures for Post-Traumatic Stress Disorder in North Korean Defectors: A Scoping Review

Ocksim Kim; Kyoung-A Kim; Sang Hui Chu

doi:10.30773/pi.2022.0363

Psychiatry Investig > Volume 20(7); 2023 > Article

Kim, Kim, and Chu: Properties of Patient-Reported Outcome Measures for Post-Traumatic Stress Disorder in North Korean Defectors: A Scoping Review

Review Article

Psychiatry Investigation 2023;20(7):593-608.

Published online: July 24, 2023

DOI: https://doi.org/10.30773/pi.2022.0363

Properties of Patient-Reported Outcome Measures for Post-Traumatic Stress Disorder in North Korean Defectors: A Scoping Review

Ocksim Kim¹

, Kyoung-A Kim²

, Sang Hui Chu³

¹Department of Nursing, Yonsei University College of Nursing and Brain Korea 21 FOUR Project, Seoul, Republic of Korea

²Department of Nursing, College of Nursing, Gachon University, Incheon, Republic of Korea

³Department of Nursing, Mo-Im Kim Nursing Research Institute, Yonsei University College of Nursing, Seoul, Republic of Korea

Correspondence: Sang Hui Chu, PhD, RN Department of Nursing, Mo-Im Kim Nursing Research Institute, Yonsei University College of Nursing, 50-1 Yonsei-ro, Seodaemun-gu, Seoul 03722, Republic of Korea Tel: +82-2-2228-3257, Fax: +82-2-2227-8302, E-mail: SHCHU@yuhs.ac

Received December 14, 2022 Revised March 24, 2023 Accepted April 16, 2023

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objective

Posttraumatic stress disorder (PTSD) is highly prevalent among North Korean defectors. This scoping review aimed to appraise the measurement properties of patient-reported outcome measures (PROMs) that assess PTSD among North Korean defectors.

Methods

A systematic search was conducted using Research Information Sharing Service, ScienceON, PubMed, and Embase from their inception up to September 15, 2022. Each measurement properties of the included PROMs were evaluated using COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) methodology. First, the methodological quality of each study on measurement properties was evaluated using the COSMIN Risk of Bias checklist. Second, the result of each single study on a measurement property was rated against the updated criteria for good measurement properties.

Results

After screening the articles, nine instruments from 40 studies were included. Among these instruments, Impact of Event ScaleRevised was the most frequently used (13 studies [38.1%]), followed by the Trauma Scale for North Korean Refugees, PTSD Checklist, and Posttraumatic Diagnostic Scale. The theoretical basis of all PROMs was secure in terms of content validity. The most frequently tested characteristics in measurement properties for internal structure were internal consistency, criterion validity, and convergent validity. No study reported sufficient structural validity as confirmatory factor analysis results with a model fit.

Conclusion

There is insufficient evidence to support the recommendation of a specific PROM for use among North Korean defectors with PTSD.

Keywords: Posttraumatic stress disorder, Patient reported outcome measures, Scoping review

INTRODUCTION

Posttraumatic stress disorder (PTSD) is a psychiatric disorder, induced when an individual is exposed to a traumatic life event, resulting in impairments in the individual’s overall daily life functioning. Difficulties in emotional control and negative changes in thinking and mood often develop into chronic PTSD (CPTSD), hindering an individual’s ability to engage in social interactions, and leading to extreme mental distress that can impede their relationships or economic life [1].

Experts use interview tools and patient-reported outcome measurements (PROMs) to diagnose PTSD symptoms. PROMs are structured and standardized self-assessment tools that assess symptoms, functional condition, and quality of life based on the patient’s direct experience, with the goal of determining treatment outcomes [2]. In PTSD treatment, PROMs provide researchers with information regarding the patient’s perception of their symptoms, and how these symptoms affect their overall function and relationships with others. Therefore, PROMs are essential communication and feedback tools for PTSD diagnosis that goes beyond reporting health and treatment-related satisfaction or feelings [3].

To allow for the clinical application of mental health PROMs, there is a need to elucidate the nature of the trauma experienced by individuals and the social culture. Since post-traumatic psychological adaptation is more dependent on how an event is perceived and accepted than on the diagnostic criteria for PTSD, it is crucial to consider the cultural characteristics related to the individual’s traumatic experience. Therefore, it is essential to assess PTSD and related symptoms from the onset of the traumatic event and to consider these factors when surveying appropriate interventions to prevent the development of CPTSD [4,5].

As of December 2022, a total of 33,882 North Korean defectors had escaped from North Korea and settled in South Korea [6]. During their escape, defectors experience both physical and mental trauma, including separation from family members, detention in labor/political prison camps, public executions, torture, beatings, punishment, and prostitution [7-10]. The lifetime prevalence of PTSD among North Korean defectors is 15.3 times higher than that among South Koreans [11].

So far various measurements have been used to study PTSD, including partial PTSD, among North Korean defectors; the diagnostic criteria were based on the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) [12]. Thus, it has been found that the prevalence rates vary according to the type or characteristics of each tool [13]. Using tools whose suitability has not been reviewed is likely to present distorted research results [14], and the degree of support for interpretation or theory of research results may be insufficient [15,16].

This scoping review aimed to identify the current use of PROMs and determine the properties of each measurement used in studies on PTSD among North Korean defectors. To identify the appropriate tools for research and clinical practice [17], the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) classification and checklist for evaluating self-reporting measurement tools were used [18]. The findings of this study could inform the selection of appropriate measurement tools for evaluating PTSD symptoms among North Korean defectors.

METHODS

Literature search strategy

A systematic literature search was conducted according to the COSMIN methodology through four databases: Research Information Sharing Service, ScienceON, PubMed, and Embase from their inception up to September 15, 2022. To identify all PROMs that have been used to detect PTSD in North Korean defectors, we used the following two search elements, the construct of interest (PTSD) and population (North Korean defectors), as follows: (((((Post-Traumatic Stress Disorder) OR (post-traumatic stress disorders)) OR (mental health trauma)) OR (psychiatry*)) OR (psychology*)) AND (((((North Korean defectors) OR (North Korean refugees)) OR (North Korean immigrants)) OR (Saetemin)) OR (North Korean emigrants)). The search term for the type of instrument and measurement properties were not applied because our study aimed to identify all PROMs that have been used for North Korean defectors. Search terms on the construct and study population of interest were reviewed by the librarian to expand the search with the appropriate terms.

Eligibility criteria

The inclusion criteria were: 1) papers published in English or Korean; 2) mental health and social adaptation papers that used PROMs to assess PTSD symptoms or major-related variables; and 3) papers involving North Korean defectors aged ≥19 years.

The exclusion criteria were: 1) papers that did not include PROMs to measure PTSD; 2) papers involving North Korean defectors aged <19 years; 3) papers involving North Korean defectors who do not live in South Korea; and 4) literature reviews, comments, qualitative research, and unpublished academic papers/research reports with unidentified original documents.

Data collection process

The data collection and selection were conducted independently by two researchers. In cases of disagreement between the researchers, the data were reviewed by a third researcher until a common consensus was reached. Retrieved data were managed using EndNote X9 (Clarivate, Philadelphia, PA, USA), a literature management program.

The process of selecting articles is presented as a flow chart of PRISMA (preferred reporting items for systematic review and meta-analysis) in Figure 1 [19]. After excluding duplicates, the electronic literature search yielded 485 articles. Initially, irrelevant studies were excluded by reviewing titles and abstracts and then 70 full-text articles were reviewed. The manual search based on the bibliographic references of the included studies yielded five additional studies. A total of 40 studies that used PROMs were reviewed in this study. Finally, we identified 9 validated PROMs, which were evaluated based on the COSMIN methodology in this study.

Evaluation of each measurement properties of the included PROMs

Each measurement’s properties of the included 9 PROMs were evaluated using the initial 2 steps of 3 substeps for performing a systematic review of COSMIN methodology [18,20].

First, the methodological quality of each study on measurement properties was evaluated using the COSMIN Risk of Bias checklist [17,20]. This checklist was developed to systematically evaluate tools that determine the PROM’s content validity, structural validity, internal consistency, cross-cultural validity/measurement invariance, reliability, measurement error, criterion validity, hypothesis testing for construct validity, and responsiveness. Content validity is a crucial measurement attribute that determines how well the PROM’s content reflects the measurement concept including relevance, comprehensiveness, and comprehensibility [17]. The internal structure refers to how the different items in the PROM are related, this step concerns an evaluation of structural validity, internal consistency, cross-cultural validity, and other forms of measurement invariance [20]. Structural validity is defined as the degree to which the scores of a measurement instrument adequately reflect the dimensionality of the construct being measured, and it can be assessed using factor analysis and item response theory (IRT)/Rasch analysis [21]. Subsequently, the remaining measurement properties (reliability, measurement error, criterion validity, hypotheses testing for construct validity, and responsiveness) were evaluated. Unlike content validity and internal structure, the evaluation of these measurement properties provides information on the quality of the scale or subscale as a whole, rather than on the item level [22]. The methodological quality of each measurement was rated as very good, adequate, doubtful, or inadequate using items in the checklist with the worst score counts principle, in which the lowest rating of any standard was taken as the methodological quality [20].

Second, the result of every single study on a measurement property was rated against the updated criteria for good measurement properties: sufficient (+), insufficient (-), or indeterminate (?) [22]. For the assessment of structural validity, the updated criteria required results related to confirmatory factor analysis (CFA) or IRT. In this study, the results from exploratory factor analysis (EFA) were evaluated by the following criteria: sufficient (+) if the identified factors explained at least 50% of the variance, indeterminate (?) if the explained variance was not reported, and insufficient (-) if the factors explained less than 50% of the variance [21]. Cronbach α was used to evaluate the property of internal consistency: sufficient (+) if the Cronbach α≥0.70 for each unidimensional scale or subscale, indeterminate (?) if the evidence was not reported, and insufficient (-) if the Cronbach α<0.70 [22]. For the assessment of reliability, the updated criteria required intraclass correlation coefficient or weighted kappa. However, in this study, reliability was assessed based on results of Pearson’s coefficient: sufficient (+) if Pearson’s r≥0.80, indeterminant (?) if Pearson’s r was not reported, and insufficient (-) if Pearson’s r<0.80 [21]. Criterion validity was also assessed using correlation with a gold standard; sufficient (+) ≥0.70, indeterminant (?) if no information was reported, and insufficient (-) <0.70 [22]. To interpret the findings of the hypothesis test, a formulation of a set of hypotheses for the expected relationship is suggested to evaluate validity and responsiveness. For the convergent validity, at least 0.50 between the PROM and the comparator instrument that intends to measure the same construct was interpreted as sufficient in this study [20]. Unlike convergence validity, the discriminant validity hypothesizes that a weak correlation between the PROM and the comparative measurement tool that measures different properties was interpreted as sufficient [23]. The known group validity test hypothesizes that there will be differences in the concept of interest scores according to the group, and empirically judged by the research team [24].

RESULTS

General characteristics of the included PROMs and studies

Table 1 presents the characteristics of 40 studies in which 13 PROMs were used to detect PTSD. The number of participants included in the studies ranged from 14 to 702 (mean age range: 25.9 [standard deviation, SD=±3.0] to 52.8 [SD= ±15.9] years) and 17 studies reported the PTSD prevalence ranged from 14.0% [25] to 71.0% [26]. The range of CPTSD prevalence was 11.3% [27] to 40.7% [9], while another study reported a lifetime prevalence of 15.3% [11]. Twenty-four studies were conducted with participants living in the community [9,11,28-44,46-48,61,62], 8 studies at Hanawon (The Settlement Support Center for North Korean defectors) [26,27,50-55], 6 studies at medical centers [25,45,48,56-59], and 2 studies were conducted using an online survey [49,60]. Out of 40 studies reviewed, only one study reported the longitudinal trajectories of the mental health of North Korean defectors [48].

However, only 9 PROMs were validated for South or North Koreans before they had used for the study: Impact of Event Scale-Revised-Korean (IES-R-K) [63,64], Impact of Event Scale-Revised-North Korean Refugees (IES-R-NKR) [65], Minnesota Multiphasic Personality Inventory-PTSD-Korean (MMPI-PTSD-K) [66], PTSD Checklist Civilian-North Korean Refugees (PCL-C-NKR) [67], PTSD Checklist-5-Korean (PCL-5-K) [68], Trauma Scale for North Korean Refugees [69], Posttraumatic stress Diagnosis Scale-Korean (PDS-K) [70], and Post-Traumatic Stress Symptom Scale for North Korean Refugees [71]. To evaluate the properties of each measurement, we excluded 4 tools that had only been translated into Korean without any validation. For example, PDS was excluded but PDS-K was included because PDS-K was validated in South Koreans. And two versions of IES-R-K and IES-R-NKR, which were translated and tested by different researchers, were included and evaluated separately (Table 2).

Characteristics of PTSD measurement tools

We found nine validated PROMs that had been used in the studies on North Korean defectors (Table 2). IES-R, MMPI-PTSD, PCL-C, PCL-5, and PDS were originally developed in foreign countries, and have been validated as South or North Korean version.

IES-R is the revised version of the IES scale (7 intrusion items and 8 avoidance items) developed by Horowitz et al. [72], comprising 22 items [73]. In IES-R, intrusion symptoms (8), avoidance symptoms (8), and hyper-arousal symptoms (6) were evaluated on a 5-point scale (0-4). MMPI-PTSD is a scale created by extracting items characteristically appearing in PTSD patients among MMPI items [74]. Subscale is divided into seven categories: emotional problems (12), memory problems (10), interpersonal problems (7), unrealistic experiences and thinking (3), impulse control problems and physical discomfort (3), concentration disorders (5), and destructive impulses (4) [66]. The PTSD Checklist (PCL) is based on the PTSD diagnostic criteria of DSM-IV. It consists of 17 items to assess re-experience (5), avoidance symptoms (7), and hyperarousal symptoms (5) on the Likert scale from 1 point (not at all) to 5 points (very much) [67,75]. There exist several versions that can be used for the general public (civilian versions, PCL-C), special occupations (specific versions, PCL-S), and military officials (military versions, PCL-M) [76]. The PCL-5 has been changed in accordance with the revised standards of DSM-V, and “negative interaction recognition and mood” has been added to “intrusion,” “avoidance,” and “alterations in addition and reactivity.” [77] PDS [78] is a tool that can diagnose post-traumatic stress and evaluate the severity of symptoms, including 17 items of re-experience (5), avoidance (7), and hypersensitivity (5), which were evaluated on the Likert scale from 0 to 3.

The Trauma Scale for North Korean Refugees [69] and the Post-Traumatic Stress Symptom Scale for North Korean Refugees [71] are PROMs that have been particularly developed and validated for North Korean defectors. Understanding both the traumatic experiences and their related symptoms will be helpful in selecting a treatment approach because the traumatic events experienced by North Korean defectors during their escape journey are quite different from those experienced by the general population. These two PROMs include a trauma experience checklist specialized to North Korean defectors such as imprisonment, torture, threats to personal security, etc. The Trauma Scale for North Korean Refugees [69] was developed for North Korean defectors in accordance with the DSM-IV diagnostic criteria, and a total of 16 items were answered on a Likert scale of 1 point to 4 points to measure PTSD symptoms. The Post-Traumatic Stress Symptom Scale-North Korean Refugees was developed to incorporate components of complex PTSD based on the DSM-IV criteria [71].

Methodological quality

The methodological quality of each study is presented in Table 3. Internal consistency was evaluated in all the studies. No study assessed responsiveness, which is a criterion for detecting changes in the constituent concept over time. No study demonstrates evidence for cross-cultural validity and measurement error.

Content validity

The relevance of the components of the content validity study and trauma was confirmed in all studies. Comprehensiveness was evaluated based on the diagnostic criteria of the DSM to measure PTSD symptoms with all the included studies reporting its values. Tools that examined the validity of the original tools in Korean studies or those involving North Korean defectors did not report comprehensiveness values. Comprehensibility was reported for four tools (IES-R-K [63], IES-R-NKR [65], Trauma Scale for North Korean Refugees [69], and Post-Traumatic Stress Symptom Scale-North Korean Refugees [71]); however, this was difficult to evaluate considering the lack of information regarding the number of individuals surveyed (whether less than 30) and whether interviews were conducted. In addition, it was described that experts participated in the tool translation process; however, there was no mention of content validity index or content validity ratio. The studies that asked participants for comprehensibility and modified the questions were given a sufficient (+) rating (Table 4).

Structural validity

In this study, there were 4 studies confirming the number of factors using principal component analysis (PCA) and the total variance was 46.9% [63], 68.8% [64], 34.9% [66], and 66.9% [68], respectively (Table 3).

One study conducted CFA, but the model fit was not reported. In the IES-R-K, the sample size was considered insufficient because the number of participants (sample size, n=105) was five times lesser than the total number of questions (22 questions×5). Model suitability for factor analysis was not described in all studies.

Internal consistency

Internal consistency indicates to what extent the subscale measures the latent variable, and Cronbach α was generally used. When the measurement tools are multidimensional, it is recommended to present Cronbach α for the subscale rather than Cronbach α for the total scale. There were a total of 4 studies that presented all the values of the subscale [63,67,70,71], and only the total value was presented in 5 studies [64-66,68,69]. All internal consistency met the criteria at Cronbach α>0.70.

Reliability

Reliability was assessed in six of the studies, by administering the same instruments to the same respondents at different times (i.e., the test-retest reliability) [63-66,68,70]. If the period was appropriate from 1 to 4 weeks and the Pearson correlation coefficient was ≥0.70, then the evidence was found to be sufficient. The studies that did not perform test-retest in this study were two studies conducted at Hanawon (an initial settlement education institution, after entering South Korea) or conmmunity [69,71] and one study conducted at a medical clinic [67].

Criterion validity

Criterion validity is defined as the degree to which the scores of a measurement instrument adequately reflect a standard [21], and using the Clinician-Administered PTSD Scale (CAPS) and Structured Clinical Interview for the DSM (SCID) as the “gold standard” measure of PTSD symptomatology [79,80]. In five studies, concurrent validity was measured and three studies showed that the correlation analysis result with the gold standard was r≥0.70. The correlation coefficient between PCL-C-NKR and CAPS was lower (r=0.47) than Beck Depression Inventory (BDI), a convergent validity measurement tool (Table 3). In addition, MMPI-PTSD and IES-R-K were used as standardization tools for the criterion for IES-R-K/PCL-C-MKR and PCL-5-K/PDS-K, respectively.

Hypothesis testing construct validity

Hypothesis tests include convergent validity, discriminant validity, and Known-groups validity [21]. In this study, depression inventory and anxiety inventory were frequently used to test convergent validity. The correlation between PTSD PROMs and depression scales [63-65,67,69,70] and anxiety scales [63,67,69,70] was found to be 0.30 to 0.83. The correlation between Brief Psychological State Inventory (BPSI), Minnesota Multiphasic Personality Inventory (MMPI) and Combat Exposure Scale-Korean (CES-K) was found to be 55, 0.38 and 0.48 [65,68,71].

The discriminant validity was tested using the State-Trait Anxiety Inventory (STAI) [64], Alcohol Use Disorders Identification Test and BPSI-Alcohol [65], and MMPI [71]. There was a low correlation of 0.06 to 0.67 (Table 3).

The Known-groups validity hypothesizes that there will be a difference in the conceptual score of interest in the group of two or more study subjects. In this study, there were four papers including PTSD/non PTSD [67-70], or PTSD/psychometric/normal groups [64,66,70] and the symptom scores of the PTSD groups were significantly higher than the non-PTSD or normal group.

Other psychometric properties

No studies were found to determine evidence for measurement error and responsiveness.

DISCUSSION

In this scoping review, we evaluated the psychological measurement properties of the PROMs used to measure PTSD in North Korean defectors. This scoping review found that 9 PTSD measurements are currently available. The psychometric properties of 9 PROMs validated in Korea were evaluated using the COSMIN checklist.

According to our results, the theoretical basis was judged to be secure in terms of content validity, as all tools were linked to the DSM and developed to target those who experienced trauma (including survivors from earthquakes, workers in emergencies, veterans from the Vietnam War, and North Korean defectors). The concept of the PTSD measurement tools identified in this study had been determined according to the diagnostic criteria of the DSM-IV or DSM-V in their original versions, and most of the tools were translated later. However, DSM-V was revised in 2013 and included the number of key symptoms of the Disorder of Extreme Stress Not Otherwise Specified (DESNOS) such as persistent erroneous blame of self or others, negative expectations about the future, persistent negative mood, and externalizing behaviors, considering CPTSD that was not available in DSM-IV [77]. This concept of CPTSD was defined by International Classification of Diseases 11th revision (ICD-11) in 2018, and classified into unique symptoms of emotional difficulty, negative self-concept, and relationship difficulties, including re-experience, avoidance, and the threat of PTSD.

North Korean defectors have experienced a combination of various types of trauma for a long time during their stay in North Korea, China, or a third country, and in the process of settling in South Korea [10,11]. North Korean defectors who experienced interpersonal trauma or life-threatening diseases were generally considered to have more severe forms of PTSD and were more likely to be classified into a dual diagnostic group of CPTSD and PTSD [9]. The type, duration, and symptoms of trauma experienced by North Korean defectors were found to be closer to CPTSD than simple PTSD [5,35,36,38,53,65]. Thus, when measuring the trauma experiences and symptoms of North Korean defectors, it is necessary to use a tool that can capture CPTSD attributes.

IES-R, PCL-C, PCL-5, PDS, and MMPI-PTSD were developed in the United States, and IES-R was validated for South Koreans [63,64] and North Korean defectors living in South Korea [65]. IES-R-NKR [65] reflects the linguistic differences between South Korea and North Korea and revises them into terms that are easy for North Korean defectors to understand. PCL-C-NKR [67] was validated for North Korean defectors who visited the medical center; however, there was no mention of verbal modifications in the process. In particular, since the differences between cutoff scores in the previous studies should be checked, and the research was conducted on North Korean defectors suspected of suffering from PTSD, this tool should be cautiously administered to the general public in the community. The differences between cutoff scores in the previous studies should be checked.

Among the selected tools, Trauma Scale for North Korean Refugees and the Post-Traumatic Stress Symptom Scale for North Korean Refugees were developed to measure PTSD of North Korean defectors. For these tools, it is to be noted that cultural and linguistic differences between South and North Korea were considered in the development process. The Trauma Scale for North Korean Refugees was developed based on the SCID, but was used more as a checklist of trauma experience than a symptom scale [30,35,50,53,60]. The trauma Scale for North Korean Refugees is beneficial for various researchers because it was converted from interview tools to PROMs. However, among the symptoms of avoidance/numbing of DSM-IV, “Feelings of detachment or estrangement from others” is not included, creating a difference in direct comparison with the results of other measurement tools.

Post-Traumatic Stress Symptom Scale for North Korean Refugees is a tool developed to measure not only PTSD but also CPTSD [71]. Post-Traumatic Stress Symptom Scale for North Korean Refugees has been adapted from the Davidson Trauma Scale. Symptoms of CPTSD such as anger, personality change, chronic guilt and responsibility, emotional control difficulties, high drug dependence, violence, somatization, and distrust of others, which are the main characteristics of DESNOS were included. The Post-Traumatic Stress Symptom Scale for North Korean Refugees was also used more as a trauma experience checklist than as a symptom scale [10,28,60]. In addition, the trauma checklist included in the measurement tool allows a more detailed understanding of trauma experiences through three categories: North Korean residence, North Korean defector process, and South Korean society.

MMPI-PTSD consists of 49 questions empirically extracted from MMPI [74], however, only 45 of its questions have been validated in South Korea [66]. Although the MMPI-PTSD has been criticized for evaluating only general psychological pain rather than specific PTSD symptoms, it has the advantage of being able to detect faking and response bias. It can also extensively evaluates the general problems of PTSD [74]. However, studies have reported different results in terms of accuracy, indicating caution for its use [81].

In the case of a tool with a predetermined structure, it is not necessary to perform an EFA that determines the number of factors [82]. Especially for foreign-origin tools, it is recommended to perform a CFA because the constituent concepts and theories have already been secured during the validation stage in other languages [20,23,83]. Therefore, it would be more appropriate to implement CFA than PCA. In most studies, only the number of factors was confirmed through PCA, and CFA was rarely conducted on the hypothesized measurement model. In a study developing Post-Traumatic Stress Symptom Scale for North Korean Refugees, CFA was conducted, and each factor correlation was reported as 0.63-0.89. The sample size was 302, which can be considered sufficient, as the minimum sample size required in the measurement model proposed by COSMIN is 200 [20]. In addition to factor analysis, it is necessary to present an index of the suitability of the measurement model. However, indices indicating the fitness of the measurement model were not mentioned.

Reliability reflects the degree of relevance and consistency of the measurement scores and can be evaluated as interrater reliability, test-retest reliability, and intra-rater reliability. The test-retest reliability is to check the stability of the measurement tool and to evaluate whether the results obtained after repeated measurements on the same subject more than once under the same conditions are consistent [84]. Many studies report test-retest reliability, and in this study, it was overall evaluated well.

As for the criterion validity, CAPS and SCID were used and the correlation was found to be good. For convergent validity for hypothesis testing, depression and anxiety measurement tools such as BDI, CES-D, and STAI were used. PTSD is known to have a high correlation with depression and anxiety. Yet, Eun et al. [63], used an anxiety measurement tool as discriminative validity. In most studies, the convergent validity was confirmed, but the discriminative validity was not accurately specified.

No studies mentioned the measurement error and responsiveness, which refers to the systematic and random error of a patient’s score that is not attributed to true changes in the construct to be measured [21]. To assess the measurement error of an outcome measurement tool, repeated measurements in stable patients are required [85]. The measurement property of responsiveness refers to the ability of PROMs to detect change over time in the construct to be measured [22]. Particularly in intervention studies, it is important to select a responsive measurement tool to indicate reactivity to treatment [84]. It has been reported that the lack of evaluation of measurement error and responsiveness was common in several literature reviews, evaluating the measurement tool in accordance with the COSMIN standard [86,87].

In addition, we found that few studies used the tools based on DESNOS [31,50], which are translated and modified without validation in South Korean [9,27], to measure CPTSD among North Korean defectors. To explain more clearly the impact of CPTSD/PTSD on the physical, psychological, and social health of North Korean defectors, it is suggested to assess CPTSD as well as PTSD using a valid measurement tool. Recently, the International Trauma Questionnaire (ITQ) [88] was newly introduced to asses core features of both PTSD and CPTSD based on the revised ICD-11 [89]. The validity of ITQ has been reported for adults who suffered from adverse childhood experiences in South Korea [89]. Since the PTSD studies for North Korean defectors so far mainly have been dealing with PTSD, it seems necessary to validate ITQ for North Korean defectors.

The followings were the limitations of this study. First, we did not review the trauma event checklist, which is the first condition for PTSD diagnosis. The trauma experience of North Korean defectors is somewhat different from that of general South Koreans, veterans, or workers in emergent settings. To measure the trauma experience unique to North Korean defectors, it is needed to evaluate the trauma event checklist and select the appropriate one for further study. Second, since this study included studies that only focused on adults, measurement tools used for children were excluded. Exposure to violence, emotional/verbal abuse, sexual abuse, and bystander behavior in childhood is known to be related to CPTSD. And there are also North Korean defectors in their childhood at risk of CPTSD. Thus, valid measurement tools to detect and manage PTSD/CPTSD early are needed for this population.

Conclusions

This scoping review found that 9 PTSD measurements are currently available for North Korean defectors, and most of the PROMs have not been evaluated on their psychometric properties thoroughly. Because all tools were linked to the DSM and developed targeting those who experienced trauma, the theoretical basis was secure in terms of content validity. However, few studies involved North Korean defectors in the process of development or validation. And the most frequently tested characteristics in measurement properties for the internal structure were internal consistency, criterion validity, and convergent validity. No study reported sufficient structural validity as CFA results with a model fit. Thus, there is insufficient evidence to support the recommendation of a specific PROM for use among North Korean defectors with PTSD. Furthermore, it is suggested that the PROM which could capture core features of both PTSD and CPTSD based on the newly revised ICD-11 should be validated for North Korean defectors.

To effectively assess the PTSD of North Korean defectors, it is imperative to employ valid and reliable measurement tools that take into account their distinct trauma experiences, cultural backgrounds, and language proficiency. This will enhance our understanding of their mental health needs and facilitate more efficient intervention and support.

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Sang Hui Chu. Data curation: Ocksim Kim, Kyoung-A Kim. Formal analysis: Ocksim Kim, Kyoung-A Kim. Investigation: Ocksim Kim, Kyoung-A Kim. Methodology: Ocksim Kim, Kyoung-A Kim. Supervision: Sang Hui Chu. Writing—original draft: Ocksim Kim, Kyoung-A Kim. Writing—review & editing: Sang Hui Chu.

Funding Statement

This research was supported by the Basic Science Research Program through the National Research Foundation (NRF) of Korea funded by the Ministry of Education (2019R1I1A2A01058746). Ocksim Kim received a scholarship from the Brain Korea 21 FOUR Project funded by the National Research Foundation (NRF) of Korea, Yonsei University College of Nursing.

Figure 1.

Flow chart for study selection. RISS, Research Information Sharing Service; PROMs, patient-reported outcome measurements.

Table 1.

Characteristics of the included instruments and studies

	Study	Sample size	Age (yr)	Spent time in South Korea (yr)	PROMs	PTSD prevalence (%)	Place
1	Hu et al., [50] (2008)	145	34	Less than 3 months	Trauma Scale for NKR (Kang, 2001)	-	Hanawon (Settlement Support Center)
1	Hu et al., [50] (2008)	145	34	Less than 3 months	CPTSD (van der Kolk, 2007)	-	Hanawon (Settlement Support Center)
2	Rho and Won [61] (2009)	105	Not reported (20-50 ≤)	-	PDS (Foa et al., 1997)	-	G city Counseling Support Center
3	Park and Ann [28] (2009)	115	33.5±12.8	-	PDS (Foa et al., 1997)	-	Seoul, Inchen
4	Lee et al., [51] (2010)	105	35.5±6.2	Less than 3 months	PDS (Foa et al., 1997)	25.7	Hanawon (Settlement Support Center)
5	Kim et al., [29] (2011)	105	Not reported (20-50 ≤)	-	PDS (Foa et al., 1997)	-	Gwangju Metropolitan City
6	Song et al., [56] (2011)	32	50.2±14.8	3.7±3.0	MMPI-PTSD-K (Kang, 1997)	-	National Medical Center
7	Choe and Son [30] (2011)	Case=6	Not reported	2.1±3.4	Trauma Scale for NKR (Kang, 2001)	-	Jeonju
7	Choe and Son [30] (2011)	Control=7	Not reported	2.1±3.4	Trauma Scale for NKR (Kang, 2001)	-	Jeonju
8	Kim and Choi [52] (2013)	Case=32	Case=37.1	Less than 3 months	IES-R-K (Eun et al., 2005)	-	Hanawon (Settlement Support Center)
8	Kim and Choi [52] (2013)	Control=38	Control=36.6	Less than 3 months	IES-R-K (Eun et al., 2005)	-	Hanawon (Settlement Support Center)
9	Hu et al., [31] (2013)	14	26.5	4.3	Trauma Scale for NKR (Kang, 2001)	-	Seoul
9	Hu et al., [31] (2013)	14	26.5	4.3	CPTSD (van der Kolk, 2007)	-	Seoul
10	Chang and Son [9] (2014)	81	35.9±13.1	2.4±0.2	PDS (Foa et al., 1997)	40.7^*	Gumi, Daegu, Gyeongbuk
10	Chang and Son [9] (2014)	81	35.9±13.1	2.4±0.2	CPTSD-Interview (Ouimette et al., 1996)	40.7^*	Gumi, Daegu, Gyeongbuk
11	Kim [27] (2014)	531	34.6±8.6	Less than 3 months	PDS (Foa et al., 1997)	14.6	Hanawon (Settlement Support Center)
11	Kim [27] (2014)	531	34.6±8.6	Less than 3 months	CPTSD- Interview (Ouimette et al., 1996)	11.3^*	Hanawon (Settlement Support Center)
12	Park et al., [33] (2015)	199	38.6±12.3	-	IES-R-K (Eun et al., 2005)	-	Community
13	Jun et al., [32] (2015)	201	38.8±12.2	4.2±2.7	IES-R-K (Eun et al., 2005)	-	Community
14	Kim and Lee [34] (2015)	78	Not reported (20-60)	-	Post-Traumatic Stress Symptom Scale-NKR (Yoon et al., 2007)	11.5^*	Seoul, Gyeonggi, Inchen
15	Lee et al., [35] (2016)	177	38.2±10.3	4.1±2.6	IES-R-K (Eun et al., 2005)	40.1	Community
16	Kim and Kim [37] (2016)	145	Not reported (20-50≤)	-	IES-R-K (Eun et al., 2005)	-	Community
17	Koo [38] (2016)	55	34.9±10.4	-	Trauma Scale for NKR (Kang, 2001)	43.6	Busan, Daegu, Seoul
18	Lee et al., [36] (2016)	45	35.44±12.1	5.2±2.7	IES-R-K (Eun et al., 2005)	-	Seoul
19	Choi et al., [39] (2017)	211	38.5±12.2	-	IES-R-K (Eun et al., 2005)	-	Seoul
20	Kim and Kim [40] (2017)	Non-PTSD=79	Non-PTSD=33.8±8.9	-	IES-R-K (Eun et al., 2005)	37.8	Hana center
20	Kim and Kim [40] (2017)	PTSD=48	PTSD=36.1±8.1	-	IES-R-K (Eun et al., 2005)	37.8	Hana center
21	Shin et al., [58] (2017)	53	PTSD=47.0±7.0	4.8±3.0	MMPI-PTSD (Keane et al., 1984)	-	National Medical Center
21	Shin et al., [58] (2017)	53	Non-PTSD=44.6±10.4	4.8±3.0	MMPI-PTSD (Keane et al., 1984)	-	National Medical Center
22	Won et al., [25] (2017)	200	47.6±11.1	-	PCL-C-NKR (Oh et al., 2014)	14.0	National Medical Center
23	Emery et al. [41] (2018)	204	Not reported	-	IES-R-K (Eun et al., 2005)	63.2	Community
24	Lee et al., [11] (2018)	300	PTSD=39.9±11.3	-	IES-R-NKR (Won et al., 2015)	15.3^**	Hana center
24	Lee et al., [11] (2018)	300	Non-PTSD=38.5±12.0	-	IES-R-NKR (Won et al., 2015)	15.3^**	Hana center
25	Kim et al., [53] (2019)	702	36.0±11.0	Less than 3 months	IES-R-K (Eun et al., 2005)	44.9	Hanawon (Settlement Support Center)
26	Lee and Kim [60] (2019)	103	25.9±3.0	5.1±3.8	Trauma Scale for NKR (Kang, 2001)	-	Online survey
27	Kim [42] (2020)	285	40.2±8.39	-	Trauma Scale for NKR (Kang, 2001)	-	Community
28	Jeon et al., [59] (2020)	38	Case=38.9	-	IES-R-K (Eun et al., 2005)	-	Myongji Hospital
28	Jeon et al., [59] (2020)	38	Control=36.9	-	IES-R-K (Eun et al., 2005)	-	Myongji Hospital
29	Chang et al., [54] (2021)	545	Not reported (35-50≤)	Less than 3 months	PDS-K (Nam et al., 2010)	6.8	Hanawon (Settlement Support Center)
30	Jeong et al., [43] (2021)	NKR=50	NK PTSD=40.3±11.5	-	IES-R-K (Lim et al., 2009)	-	Community
		SK=51	NK control=30.9±8.1
		SK=51	SK=36.6±11.2
31	Kim and Jhone [44] (2021)	300	Normal=56.6±18.8	7.3±4.4	PCL-C-NKR (Oh et al., 2014)	51.7	2017 National Human Rights Commission
			PTSD=49.5±18.1
			Depression=57.9±20.3
			Comorbidity=52.9±15.9
32	Lee et al., [46] (2021)	300	Not reported (19-60≤)	6.8±4.4	PCL-C-NKR (Oh et al., 2014)	49.3	Community
33	Lee et al., [47] (2021)	NKR=32	NKR=33.8±14.3	-	IES-R-K (Eun et al., 2005)	-	Community
33	Lee et al., [47] (2021)	SK=39	SK=35.0±11.1	-	IES-R-K (Eun et al., 2005)	-	Community
34	Lee et al., [45] (2021)	34	43.8±9.4	-	PCL-C-NKR (Oh et al., 2014)	-	National Medical Center
35	Park et al., [48] (2021)	300	38.7±11.9	-	IES-R-NKR (Won et al., 2015)	-	Hana centers
36	Shim et al., [57] (2021)	34	NKR=43±9.4	-	PCL-C-NKR (Oh et al., 2014)	-	National Medical Center
36	Shim et al., [57] (2021)	34	SK=41±9.9	-	PCL-C-NKR (Oh et al., 2014)	-	National Medical Center
37	Yun and Kim [26] (2021)	642	Repatriated=38.0±10.0	Less than 3 months	IES-R-K (Eun et al., 2005)	Repatriated= 71.0	Hanawon (Settlement Support Center)
37	Yun and Kim [26] (2021)	642	Not Repatriated=35.6±9.0	Less than 3 months	IES-R-K (Eun et al., 2005)	Not repatriated= 60.0	Hanawon (Settlement Support Center)
38	Lee et al., [49] (2022)	212	41.5±11.9	-	PCL-5-K (Kim et al., 2017)	43.9	Online survey
39	Lee [62] (2022)	299	52.8±15.9	-	PCL-C-NKR (Oh et al., 2014)	-	2017 National Human Rights Commission
40	Kim et al., [55] (2022)	627	Not reported (20-60≤)	Less than 3 months	IES-R-K (Eun et al., 2005)	-	Hanawon (Settlement Support Center)

Values of age are presented as mean±standard deviation, mean, or mean (range).

^* CPTSD;

^** lifetime prevalence.

PROMs, patient-reported outcome measurements; PTSD, posttraumatic stress disorder; CPTSD, complex PTSD; PDS, Posttraumatic Diagnosis Scale; MMPI-PTSD-K, Minnesota Multiphasic Personality Inventory-PTSD-Korean; IES-R-K, Impact of Event Scale Revised-Korean; NKR, North Korean Refugees; PCL-C-NKR, PTSD Checklist Civilian-North Korean Refugees; IES-R-NKR, Impact of Event Scale Revised-North Korean Refugees; PDS-K, PDS-Korean; NK, North Korean; SK, South Korean

Table 2.

Characteristics of included PTSD PROMs

PROMs	Language	Total (N)	Population		Instrument administration
PROMs	Language	Total (N)	Age (yr)	Gender (%)	Setting	Cutoff score (sensitivity, specificity)	Original Translated version
IES-R-K (Eun et al., 2005) [63]	South Korean	105	10-50	Men=42.9	University hospital, General hospital, Private hospital	24/25 (1.00, 0.60)	Translated
IES-R-K (Eun et al., 2005) [63]	South Korean	105	10-50	Women=57.1	University hospital, General hospital, Private hospital	24/25 (1.00, 0.60)	Translated
IES-R-K (Lim et al., 2009) [64]	South Korean	PTSD=93	44.9±15.7	Men=51.6	St. Mary’s Hospital, The Catholic University of Korea, and the 17 other collaborating centers	22 (0.95, 0.80)	Translated
		psychiatric=73	44.6±14.6	Men=37.5
		Normal=88	43.0±13.2	Men=56.8
IES-R-NKR (Won et al., 2015) [65]	North Korean defectors	207	Men: 38.0±11.9	Men=21.7	Hana center (Settlement Help Agency)	-	Translated
IES-R-NKR (Won et al., 2015) [65]	North Korean defectors	207	Women: 41.1±13.5	Women=78.3	Hana center (Settlement Help Agency)	-	Translated
MMPI-PTSD-K (Kang, 1997) [66]	South Korean	PTSD=68	N/R	Men=61.2	Mental Health Hospital	17	Translated
		Psyciatric=68		Women=38.3
		Normal=122		Women=38.3
PCL-C-NKR (Oh et al., 2014) [67]	North Korean defectors	PTSD=32	PTSD: 70.1±2.7	Men=13.0	National Medical Center	56 (0.88, 0.50)	Translated
PCL-C-NKR (Oh et al., 2014) [67]	North Korean defectors	Non-PTSD=37	Non-PTSD: 70.3±3.4	Women=87.0	National Medical Center	56 (0.88, 0.50)	Translated
PCL-5-K (Kim et al., 2017) [68]	South Korean	PTSD=42	PTSD: 42.2±9.4		Veterans Health Service Medical Center (Vietnam War participants)	37 (0.88, 0.96)	Translated
PCL-5-K (Kim et al., 2017) [68]	South Korean	Non-PTSD=162	Non-PTSD: 47.2±12.2			37 (0.88, 0.96)	Translated
PDS-K (Nam et al., 2010) [70]	South	PTSD=21	PTSD: 47.6±13.5	Men=74.4	Vietnam War participants, Combat police, Medical centers, Sexual Violence Relief Center, Traffic Disabled Association	20 (1.00, 0.84)	Translated
PDS-K (Nam et al., 2010) [70]	Korean	Subthreshold=26	Subthreshold: 41.1±20.1	Women=25.5		20 (1.00, 0.84)	Translated
Post-Traumatic Stress Symptom Scale-NKR (Yoon et al., 2007) [71]	North Korean defectors	PTSD=46	31.6	Men=26.7	Hanawon (Settlement Education Agency)	-	Original
		Psyciatric=29		Women=72.3
		Normal= 31		Unresponsive=1.0
Trauma Scale for NKR (Kang, 2001) [69]	North Korean defectors	PTSD=26	35.2	Men=65.2	Hanawon (Settlement Education Agency)	17 (0.69, 0.64)	Original
		Non-PTSD=69		Women=29.5
		Non-PTSD=69		Unresponsive=5.3

Values are presented as mean±standard deviation, number, or percentage. PTSD, posttraumatic stress disorder; PROMs, patient-reported outcome measurements; IES-R-K, Impact of Event Scale Revised-Korean; IES-R-NKR, Impact of Event Scale Revised-North Korean Refugees; MMPI-PTSD-K, Minnesota Multiphasic Personality Inventory-PTSD-Korean; PCL-C-NKR, PTSD Checklist civilian version-North Korean Refugees; PCL-5-K, PTSD Checklist-5-Korean; PDS-K, Posttraumatic Diagnostic Scale-Korean; N/R, not reported

Table 3.

Summary of results for measurement properties for each PROM

PROMs	Content validity			Structural validity	Internal consistency		Reliability	Criterion validity	Hypothesis testing
PROMs	R	Cv	Cb	Structural validity	Internal consistency		Reliability	Criterion validity	Convergent validity	Discriminative validity	Known-groups validity
IES-R-K (Eun et al., 2005) [63]	○	○	○	4 factors: PCA confirmed: oblique rotation (N=105)	Cronbach α:		2 week (N=N/R)	CAPS: r=0.69	BDI: r=0.61
						Avoidance=0.70	r=0.89	CAPS: r=0.69	STAI-I: r=0.59
						Hyperarousal=0.87	Spearman-Brown coefficient=0.71	MMPI-PTSD: r=0.66	STAI-II: r=0.54
				Total variance 46.9%		Intrusion=0.63
				Total variance 46.9%		Total scale=0.83
IES-R-K (Lim et al., 2009) [64]	○	○		4 factors: PCA confirmed: orthogonal rotation (N=254)	Cronbach α:		2 weeks (N=51)	CAPS: r=0.92	BDI: r=0.81	STAI-S: r=0.30	a: PTSD
						Total scale=0.93	Cronbach α=0.93			STAI-T: r=0.67	b: Psychiatric
							Cronbach α=0.93				c: Normal
				Total variance 68.8%			r=0.91				a>b, a>c (p<0.001)
IES-R-NKR (Won et al., 2015) [65]	○	○	○		Cronbach α:		1week (N=143)	BPSI-NKR-PTSD:	CES-D-NKR:	AUDIT-NK: r=0.15
						Total scale=0.94	Men: r=0.46	Men: r=0.72	Men: r=0.48	BPSI-Alcohol: r=0.17
							Women: r=0.75	Women: r=0.71	Women: r=0.64
							Women: r=0.75	Women: r=0.71	BPSI-Depression: r=0.55
MMPI-PTSD-K (Kang, 1997) [66]	○	○		7 factors PCA confirmed: varimax rotation (N=266)	Cronbach alpha:		1week (N=122)	Mississippi-PTSD: r=0.77			a: PTSD
						Total scale=0.86-0.90	r=0.83				b: Psychiatric
						PTSD=0.90					c: Normal
				Total variance 34.9%		Psyciatric=0.87					a>b, a>c (p<0.01)
				Total variance 34.9%		Normal=0.86					a>b, a>c (p<0.01)
PCL-C-NKR (Oh et al., 2014) [67]	○	○			Cronbach α:			CAPS: r=0.47	BDI: r=0.83		a: PTSD
						Re-experience=0.89		MMPI-PTSD: r=0.61	STAI-T: r=0.73		b: Non-PTSD
						Avoidance=0.87			STAI-S: r=0 .61		a>b (p<0.01)
						Hyperarousal=0.81
						Total scale=0.93
PCL-5-K (Kim et al., 2017) [68]	○	○		4 factors PCA confirmed: varimax rotation (N=204)	Cronbach α:		1 month over (N=30)	SCID: r=0.82	CES-K: r=0.48		a: PTSD
				4 factors PCA confirmed: varimax rotation (N=204)		Total scale=0.97	Cronbach α=0.96	IES-R-K: r=0.87			b: Non-PTSD
				Total variance 66.9%		Total scale=0.97	r=0.96	IES-R-K: r=0.87			a>b (p<0.001)
PDS-K (Nam et al., 2010) [70]	○	○			Cronbach α:		2 week (N=12)	SCID: r=0.84	BDI: r=0.69		a: PTSD
						Re-experience=0.90	r=0.81	SCID: r=0.84	BAI: r=0.69		b: Non-PTSD
						Avoidance=0.87		IES-R-K: r=0.70			a>b (p<0.001)
						Hyperarousal=0.81
						Total scale=0.95
Post-Traumatic Stress Symptom Scale-NKR (Yoon et al., 2007) [71]	○	○	○	4 factors CFA confirmed: Medel fit N/R (N=206)	Cronbach α:			Mississippi-PTSD: r=0.65	MMPI:	MMPI:	a: PTSD
						Re-experience=0.84			Depression: r=0.38	Masculinity/Femininity: r=0.06	b: Psychiatric
						Avoidance=0.81			Paranoia: r=0.31	Hypomania: r=0.08	c: Normal
						Hyperarousal=0.89			Psychasthenia: r=0.43		a>b, a>c (p<0.05)
						Complex PTSD=0.80			Schizophrenia: r=0.45
						Total scale=0.93			Schizophrenia: r=0.45
Trauma Scale for NKR (Kang, 2001) [69]	○	○	○		Cronbach α:				CES-D: t=2.34 (p<0.05)		a: PTSD
						Total scale=0.87			STAI: t=2.94 (p<0.01)		b: Non-PTSD
						Total scale=0.87			STAI: t=2.94 (p<0.01)		a>b

PTSD, posttraumatic stress disorder; PROMs, patient-reported outcome measurements; R, relevance; Cv, comprehensiveness; Cb, comprehensibility; IES-R-K, Impact of Event Scale Revised Korean; IES-R-NKR, Impact of Event Scale Revised-North Korean Refugees; MMPI-PTSD-K, Minnesota Multiphasic Personality Inventory-PTSD-Korean; PCL-C-NKR, PTSD Checklist civilian version-North Korean Refugees; PCL-5-K, PTSD Checklist-5-Korean; PDS-K, Posttraumatic Diagnostic Scale-Korean; PCA, principal component analysis; CFA, confirmatory factor analysis; CAPS, Clinician-Administered PTSD Scale; BPSI, Brief Psychological State Inventory; MMPI, Minnesota Multiphasic Personality Inventory; CES-D, Center for the Epidemiological Studies of Depression; STAI, State-Trait Anxiety Inventory; STAI-T, State-Trait Anxiety Inventory-Trait version; STAI-S, State-Trait Anxiety Inventory-State version; DSM, the Diagnostic and Statistical Manual of Mental Disorders; SCID-5, Structured clinical interview for DSM-5; BDI, Beck Depression Inventory; AUDIT, Alcohol Use Disorders Identification Test; CES-K, Combat Exposure Scale-Korean; NK, North Korean; N/R, not reported

Table 4.

Methodological quality and results rating of each PROM

PROMs	Content validity	Structural validity		Internal consistency		Reliability		Convergent validity		Hypothesis testing
	Content validity	Structural validity		Internal consistency		Reliability		Convergent validity		Criterion validity		Discriminative validity		Known-groups validity
	Results (rating)	MQ	Results (rating)	MQ	Results (rating)	MQ	Results (rating)	MQ	Results (rating)	MQ	Results (rating)	MQ	Results (rating)	MQ	Results (rating)
IES-R-K (Eun et al., 2005) [63]	+	Adequate	-	Very good	+	Adequate	+	Very good	-	Very good	+	N/R	N/R	N/R	N/R
IES-R-K (Lim et al., 2009) [64]	±	Adequate	+	Doubtful	+	Adequate	+	Very good	+	Very good	+	Very good	-	Very good	+
IES-R-NKR (Won et al., 2015) [65]	+	N/R	N/R	Doubtful	+	Adequate	+	Inadequate	+	Very good	+	Very good	+	N/R	N/R
MMPI-PTSD-K (Kang, 1997) [66]	±	Adequate	-	Very good	+	Adequate	+	Inadequate	+	N/R	N/R	N/R	N/R	Very good	+
PCL-C-NKR (Oh et al., 2014) [67]	±	N/R	N/R	Very good	+	N/R	N/R	Very good	-	Very good	+	N/R	N/R	Very good	+
PCL-5-K (Kim et al., 2017) [68]	±	Adequate	+	Doubtful	+	Adequate	+	Very good	+	Very good	-	N/R	N/R	Very good	+
PDS-K (Nam et al., 2010) [70]	±	N/R	N/R	Very good	+	Adequate	+	Very good	+	Very good	+	N/R	N/R	Very good	+
Post-Traumatic Stress Symptom Scale-NKR (Yoon et al., 2007) [71]	+	Very good	?	Very good	+	N/R	N/R	Inadequate	-	Very good	-	Very good	+	Very good	+
Trauma Scale for NKR (Kang, 2001) [69]	+	N/R	N/R	Doubtful	+	N/R	N/R	N/R	N/R	Very good	+	N/R	N/R	Very good	+

“+”: Sufficient. “−”: Insufficient. “?”: Indeterminate. “±”: Inconsistent. PTSD, posttraumatic stress disorder; PROMs, patient-reported outcome measurements; MQ, methodological quality; IES-R-K, Impact of Event Scale Revised-Korean; IES-R-NKR, Impact of Event Scale Revised-North Korean Refugees; MMPI-PTSD-K, Minnesota Multiphasic Personality Inventory-PTSDKorean; PCL-C-NKR, PTSD Checklist civilian version-North Korean Refugees; PCL-5-K, PTSD Checklist-5-Korean; PDS-K, Posttraumatic Diagnostic Scale-Korean; NKR, North Korean refugees; N/R, not reported

REFERENCES

1. Seo Y, Park J, Cho AR. Review of assessment inventories for clinical disorders and symptoms caused by mass trauma events. Korean J Couns 2015;16:557-580.

2. Kendrick T, El-Gohary M, Stuart B, Gilbody S, Churchill R, Aiken L, et al. Routine use of patient reported outcome measures (PROMs) for improving treatment of common mental health disorders in adults. Cochrane Database Syst Rev 2016;7:CD011119

3. Black N. Patient reported outcome measures could help transform healthcare. BMJ 2013;346:f167

4. Foa EB, Keane TM, Friedman MJ, Cohen JA. Effective treatments for PTSD: practice guidelines from the International Society for Traumatic Stress Studies (2nd ed). New York: Guilford Press; 2009.

5. Ursano RJ, Bell C, Eth S, Friedman M, Norwood A, Pfefferbaum B, et al. Practice guideline for the treatment of patients with acute stress disorder and posttraumatic stress disorder. Am J Psychiatry 2004;161(11 Suppl):3-31.

6. South Korean Ministry of Unification. Major statistics in inter-Korean relations [Internet]. Available at: http://unikorea.go.kr/unikorea/business/statistics/. Accessed January 28, 2023.

7. Jeon WT, Min SK, Lee MH, Lee ES. Adjustment of North Korean defectors in South Korea. J Korean Neuropsychiatr Assoc 1997;36:145-161.

8. Kim JY, Choi KH, Chae JH. Risk factors of child maltreatment among North Korean refugee families. Korean J Soc Welf Stud 2012;43:267-293.

9. Chang M, Son E. Complex PTSD symptoms and psychological problems of the North Korean defectors. Korean J Health Psychol 2014;19:973-999.

10. Cho YA, Kim YH. Predictors of mental health risks in newly resettled North Korean refugee women. Korean J Woman Psychol 2010;15:509-527.

11. Lee KE, An JH, Kim DE, Moon CS, Hong JP. Clinical characteristics of post-traumatic stress disorder among North Korean defectors. Anxiety Mood 2018;14:80-87.

12. Ko DK, Choi JH, Jeon YW, Kim BC. The prevalence of posttraumatic stress disorder among North Korean defectors. Anxiety Mood 2010;6:125-130.

13. Lee SH, Kim SJ, Yoo SY, Jeon JY. Development of psychiatric treatment model for North Korean defectors [Internet] Avaliable at: https://scienceon.kisti.re.kr/srch/selectPORSrchReport.do?cn=TRKO201300031899. Accessed October 16, 2022.

14. Cho E, Choi M, Kim EY, Yoo IY, Lee NJ. Construct validity and reliability of the Korean version of the practice environment scale of nursing work index for Korean nurses. J Korean Acad Nurs 2011;41:325-332.

15. American Educational Research Association; American Psychological Association. National Council on Measurement in Education. Standards for educational and psychological testing (1st ed). Washington DC: American Educational Research Association; 1999.

16. Lee K, Shin S. Validity of instrument development research in Korean nursing research. J Korean Acad Nurs 2013;43:697-703.

17. Terwee CB, Prinsen CAC, Chiarotto A, Westerman MJ, Patrick DL, Alonso J, et al. COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Qual Life Res 2018;27:1159-1170.

18. Mokkink LB, de Vet HCW, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res 2018;27:1171-1179.

19. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097

20. Mokkink LB, Prinsen C, Patrick DL, Alonso J, Bouter LM, de Vet HC, et al. COSMIN methodology for systematic reviews of patient-reported outcome measures (PROMs): user manual. Amsterdam: COSMIN, 2018, p.6-63.

21. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for healthrelated patient-reported outcomes. J Clin Epidemiol 2010;63:737-745.

22. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018;27:1147-1157.

23. Ji E, Kim M. Development and practice of social welfare measurement. Seoul: Hakjisa, 2015, p.231-247.

24. Lee EH. Psychometric properties of an instrument 3: convergent, discriminant, known-groups, and criterion validity. Korean J Women Health Nurs 2021;27:176-179.

25. Won SD, Lee SH, Kim S, Jun JY, Shin SS. Somatic symptoms as mediating factors between the perceived social support and the health-related quality of life in North Korean defectors. Psychiatry Investig 2017;14:407-412.

26. Yun M, Kim E. Torture experiences and mental health problems among refugees: the impact of repatriation to North Korea on mental disorders. J Aggress Maltreat Trauma 2021;30:1327-1346.

27. Kim HK. Differences in the symptoms of complex PTSD and PTSD in North Korean defectors by trauma type. Korean Soc Sci Rev 2014;4:65-92.

28. Park CO, Ahn HN. The relationship between interpersonal trauma exposure, experiential avoidance, forgiveness and posttraumatic stress disorder symptoms in North Korean refugee. Korean J Couns 2009;10:1891-1905.

29. Kim CG, Kim EY, Rho IS. The relationship between North Korean refugees’ posttraumatic stress and social adjustment: the mediating and moderating effects of self-esteem (focus on Gwangju metropolitan city). J Korean Data Anal Soc 2011;13:731-743.

30. Choe HO, Son CN. The effects of the Korean Mindfulness-Based Stress Reduction (K-MBSR) program on posttraumatic stress disorder symptoms, experiential avoidance, and shame in North Korean defectors. Korea J Health Psychol 2011;16:469-482.

31. Hu SH, Park E, Park J, Jung T. Social adaptation of college students from North Korea: a case study with focus on complex-PTSD. Korean J Stress Res 2013;21:193-202.

32. Jun JY, Lee YJ, Lee SH, Yoo SY, Song J, Kim SJ. Association between defense mechanisms and psychiatric symptoms in North Korean refugees. Compr Psychiatry 2015;56:179-187.

33. Park J, Jun JY, Lee YJ, Kim S, Lee SH, Yoo SY, et al. The association between alexithymia and posttraumatic stress symptoms following multiple exposures to traumatic events in North Korean refugees. J Psychosom Res 2015;78:77-81.

34. Kim HN, Lee JH. The use of music for alleviating PTSD symptoms among North Korean resettlers. J Music Hum Behav 2015;12:79-97.

35. Lee YJ, Jun JY, Lee YJ, Park J, Kim S, Lee SH, et al. Insomnia in North Korean refugees: association with depression and post-traumatic stress symptoms. Psychiatry Investig 2016;13:67-73.

36. Lee YJ, Jun JY, Park J, Kim S, Gwak AR, Lee SH, et al. Effects of psychiatric symptoms on attention in North Korean refugees. Psychiatry Investig 2016;13:480-487.

37. Kim YA, Kim MY. Factors influencing posttraumatic growth of North Korean defectors in South Korea. J Korea Acad Ind Coop Soc 2016;17:332-338.

38. Koo JH. Information-seeking within negative affect: lessons from North Korean refugees’ everyday information practices within PTSD. J Korean Soc Libr Inf Sci 2016;50:285-312.

39. Choi Y, Lim SY, Jun JY, Lee SH, Yoo SY, Kim S, et al. The effect of traumatic experiences and psychiatric symptoms on the life satisfaction of North Korean refugees. Psychopathology 2017;50:203-210.

40. Kim HS, Kim HK. Difference of menstrual problems according to post traumatic stress disorder among North Korean woman defectors in South Korea. Korean J Stress Res 2017;25:294-298.

41. Emery CR, Yoo J, Lieblich A, Hansen R. After the escape: physical abuse of offspring, posttraumatic stress disorder, and the legacy of political violence in the DPRK. Violence Against Women 2018;24:999-1022.

42. Kim S. Mediating effect of post-traumatic stress disorder in the relationship between trauma experience and psychological well-being of North Korean defectors. J Emot Behav Disord 2020;36:201-218.

43. Jeong H, Lee YJ, Kim N, Jeon S, Jun JY, Yoo SY, et al. Increased medial prefrontal cortical thickness and resilience to traumatic experiences in North Korean refugees. Sci Rep 2021;11:14910

44. Kim J, Jhone JH. The relationship between PTSD-depression comorbidity and suicide attempts in North Korean refugees. Ment Health Soc Work 2021;49:56-78.

45. Lee MS, Seo YE, Mok YE, Lee SH. Heart rate variability after treatment for depression in North Korean defectors. Appl Psychophysiol Biofeedback 2021;46:11-18.

46. Lee SH, Noh JW, Kim KB, Lee HW, Jun JY, Lee WW. The effects of inhumane treatment in North Korean detention facilities on the posttraumatic-stress disorder symptoms of North Korean refugees. Psychiatry Investig 2021;18:688-694.

47. Lee J, Jeon S, Kim S, Seo Y, Park J, Lee YJ, et al. Polysomnographic sleep and attentional deficits in traumatized North Korean refugees. Nat Sci Sleep 2021;13:635-645.

48. Park SH, Hong JP, An JH, Lee HR, Kim MH, Chang H. Longitudinal trajectories of mental health among North Korean defectors: a fouryear follow-up study of loneliness, depression, and life satisfaction. Anxiety Mood 2021;17:49-58.

49. Lee MK, Kim O, Kim KA, Chu SH. Factors associated with posttraumatic growth among North Korean defectors in South Korea. Sci Rep 2022;12:3989

50. Hu SH, Choi YJ, Jung TY. Diagnosis of complex post traumatic stress disorder for North Korean refugees. Korean J Stress Res 2008;16:379-386.

51. Lee IS, Park HR, Park HJ, Park YH. Relationships between parenting behavior, parenting efficacy, adaptation stress and post traumatic stress disorder among mothers who defected from North Korean. Child Health Nurs Res 2010;16:360-368.

52. Kim SH, Choi BN. The effectiveness of PTSD program of North Korean refugees: for North Korean female refugees. Korean J Women Psychol 2013;18:533-548.

53. Kim E, Yun M, Jun JY, Park WS. Pre-migration trauma, repatriation experiences, and PTSD among North Korean refugees. J Immigr Minor Health 2019;21:466-472.

54. Chang SY, Jeon JY, Maeng SR, Kim HY, Bae JN, Lee JS, et al. Prevalence of posttraumatic stress disorder among North Korean defectors during preparation for early settlement. Anxiety Mood 2021;17:67-72.

55. Kim E, Yun M, von Denkowski C. Violence against North Korean refugee women: doubly victimized by repatriation and premigration traumatic experiences. Violence Against Women 2022;28:2424-2447.

56. Song BA, Yoo SY, Kang HY, Byeon SH, Shin SH, Hwang EJ, et al. Posttraumatic stress disorder, depression, and heart-rate variability among North Korean defectors. Psychiatry Investig 2011;8:297-304.

57. Shim SS, Lee SH, Lee JB, Seo YE, Lee HJ. A study on the relationship between mental health variables and physical activity variables in the clinical group of North Korean defectors: a pilot study. J Korean Neuropsychiatr Assoc 2021;60:204-212.

58. Shin JE, Choi CH, Lee JM, Kwon JS, Lee SH, Kim HC, et al. Association between memory impairment and brain metabolite concentrations in North Korean refugees with posttraumatic stress disorder. PLoS One 2017;12:e0188953

59. Jeon S, Lee YJ, Park I, Kim N, Kim S, Jun JY, et al. Resting state functional connectivity of the thalamus in North Korean refugees with and without posttraumatic stress disorder. Sci Rep 2020;10:3194

60. Lee YJ, Kim HW. Association of traumatic events, post traumatic stress disorder and sexual autonomy among female university students of North Korean defectors. Korean J Women Health Nurs 2019;25:46-59.

61. Rho IS, Won JS. The relationship of depression, anxiety, post-traumatic stress, self-esteem, and social adjustment among North Korean refugees. J Korean Acad Psych Mental Health Nurs 2009;18:69-77.

62. Lee JH. The effects of trauma types at pre-migration, transit, and postmigration stages on depression and PTSD among North Korean refugees in South Korea. Challenges 2022;13:31

63. Eun HJ, Kwon TW, Lee SM, Kim TH, Choi MR, Cho SJ. A study on reliability and validity of the Korean version of impact of event scale-revised. J Korean Neuropsychiatr Assoc 2005;44:303-310.

64. Lim HK, Woo JM, Kim TS, Kim TH, Choi KS, Chung SK, et al. Reliability and validity of the Korean version of the Impact of Event ScaleRevised. Compr Psychiatry 2009;50:385-390.

65. Won SD, Lee SH, Hong JP, Jun JY, Han JM, Shin MN, et al. A study on reliability and validity of the Impact of Event Scale-Revised-North Korea (IES-R-NK). J Korean Neuropsychiatr Assoc 2015;54:97-104.

66. Kang YS. A comparative study of the Mississippi scale and MMPIPTSD scale to diagnose PTSD [dissertation] Jeonju, Jeonbuk National University. 1997.

67. Oh SI, Won SD, Lee SH, Yoo SY, Kim HC, Kim HJ. Reliability and validity of the Korean version of the PTSD checklist civilian version in North Korean defectors. J Korean Neuropsychiatr Assoc 2014;53:410-417.

68. Kim JW, Chung HG, Choi JH, So HS, Kang SH, Kim DS, et al. Psychometric properties of the Korean version of the PTSD checklist-5 in elderly Korean veterans of the Vietnam war. Anxiety Mood 2017;13:123-131.

69. Kang SR. Development of trauma scale for North Korean Refugee [dissertation] Seoul, Yonsei University. 2001.

70. Nam BR, Kwon HI, Kwon JH. Psychometric qualities of the Korean version of the Posttraumatic Diagnosis Scale (PDS-K). Korean J Clin Psychol 2010;29:147-167.

71. Yoon YS, Kim HA, Han SY. Validation and development of a post-traumatic stress symptom scale for dislocated North Koreans in South Korea. Korean J Couns Psychother 2007;19:693-718.

72. Horowitz M, Wilner N, Alvarez W. Impact of event scale: a measure of subjective stress. Psychosom Med 1979;41:209-218.

73. Weiss D, Marmar C. The Impact of Event Scale—Revised. In: Wilson J, Keane T, editor. Assessing psychological trauma and PTSD. New York: Guilford Press, 1997, p. 399-411.

74. Keane TM, Malloy PF, Fairbank JA. Empirical development of an MMPI subscale for the assessment of combat-related posttraumatic stress disorder. J Consult Clin Psychol 1984;52:888-891.

75. Weathers FW, Litz BT, Herman DS, Huska JA, Keane TM. The PTSD Checklist (PCL): reliability, validity, and diagnostic utility. Proceedings of the 9th Annual Meeting of the International Society for Traumatic Stress Studies; 1993 Oct 23-25; San Antonio, TX, USA: ISTSS; 1993.

76. Park S, Jeong HS, Im JJ, Jeon Y, Ma J, Choi Y, et al. Reliability and validity of the Korean version of the post-traumatic stress disorder checklist in public firefighters and rescue workers. Korean J Biol Psychiatry 2016;23:29-36.

77. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (5th ed). Arlington, VA: American Psychiatric Association; 2013.

78. Foa EB, Cashman L, Jaycox L, Perry K. The validation of a self-report measure of posttraumatic stress disorder: the Posttraumatic Diagnostic Scale. Psychol Assess 1997;9:445-451.

79. Hunt JC, Chesney SA, Jorgensen TD, Schumann NR, deRoon-Cassini TA. Exploring the gold-standard: evidence for a two-factor model of the Clinician Administered PTSD scale for the DSM-5. Psychol Trauma 2018;10:551-558.

80. Shabani A, Masoumian S, Zamirinejad S, Hejri M, Pirmorad T, Yaghmaeezadeh H. Psychometric properties of Structured Clinical Interview for DSM-5 Disorders-Clinician Version (SCID-5-CV). Brain Behav 2021;11:e01894

81. Weathers FW, Keane TM, Foa EB. Assessment and diagnosis of adults. In: Foa EB, Keane TM, Friedman MJ, Cohen JA, editor. Effective treatments for PTSD: practice guidelines from the International Society for Traumatic Stress Studies (2nd ed). New York: Guilford Press, 2009, p. 23-61.

82. Lee EH, Kang EH, Kang HJ. Evaluation of studies on the measurement properties of self-reported instruments. Asian Nurs Res (Korean Soc Nurs Sci) 2020;14:267-276.

83. Lee EH. Psychometric properties of an instrument 2: structural validity, internal consistency, and cross-cultural validity/measurement invariance. Korean J Women Health Nurs 2021;27:69-74.

84. Lee EH. Psychometric property of an instrument 4: reliability and responsiveness. Korean J Women Health Nurs 2021;27:275-277.

85. Mokkink LB, Boers M, van der Vleuten CPM, Bouter LM, Alonso J, Patrick DL, et al. COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study. BMC Med Res Methodol 2020;20:293

86. Lee EH, Moon SH, Cho MS, Park ES, Kim SY, Han JS, et al. The 21-item and 12-item versions of the Depression Anxiety Stress Scales: psychometric evaluation in a Korean population. Asian Nurs Res (Korean Soc Nurs Sci) 2019;13:30-37.

87. Köberich S, Kato NP, Kugler C, Strömberg A, Jaarsma T. Methodological quality of studies assessing validity and reliability of the European Heart Failure Self-care Behaviour Scale: a systematic review using the COSMIN methodology. Eur J Cardiovasc Nurs 2021;20:501-512.

88. Cloitre M, Shevlin M, Brewin CR, Bisson JI, Roberts NP, Maercker A, et al. The international trauma questionnaire: development of a selfreport measure of ICD-11 PTSD and complex PTSD. Acta Psychiatr Scand 2018;138:536-546.

89. Choi H, Lee W, Hyland P. Factor structure and symptom classes of ICD-11 complex posttraumatic stress disorder in a South Korean general population sample with adverse childhood experiences. Child Abuse Negl 2021;114:104982