Psychiatry Investig Search


Psychiatry Investig > Volume 15(1); 2018 > Article
Kim, Shin, Lee, Lee, Yoo, Kim, Kim, Kim, and Kim: Reliability and Validity of a New Comprehensive Tool for Assessing Challenging Behaviors in Autism Spectrum Disorder



The purpose of this study was to examine the validity and reliability of the Korean Comprehensive Scale for the Assessment of Challenging Behavior in Developmental Disorders (K-CSCB).


In total, the parents of 189 patients with autism spectrum disorder (ASD) and 168 controls completed the K-CSCB, the Behavior Problems Inventory (BPI) and Child Behavior Checklist (CBCL). The reliability and validity of the K-CSCB was investigated.


The K-CSCB was found to be a reliable instrument (Cronbach's α=0.97). There was a significant difference between the ASD and control groups in all subscale scores. Scores on the K-CSCB subscales were significantly correlated with those on the BPI and CBCL. The diagnostic validity was 97.7%, and the cut-off score with the highest sensitivity and specificity was 12.5 points.


The K-CSCB is the first tool in Korean to assess problematic behavior in individuals with ASD, and this study shows that it is a valid and reliable instrument. We expect the K-CSCB to be widely used in clinical and research settings.


The number of individuals diagnosed with autism spectrum disorder (ASD) has been increasing, with estimates ranging from 0.07% to 2.64%.1234 ASD is a developmental disorder characterized by impairment in social communication and restricted/repetitive behaviors.5 Although not included in the core symptoms of ASD, behavioral problems, including aggression, temper tantrums, non-compliance, and self-injurious behaviors, are burdensome to caregivers and commonly require intervention.6 Their presence can be a major impediment to activities, socialization, and other learning opportunities and can persist into adolescence and adulthood, affecting the long-term prognosis.7 A previous study reported that one-third of children aged 1.5 to 5.8 years showed problematic behaviors in the clinically significant range.8 Some studies have reported that over 90% of patients with ASD exhibit some form of challenging behavior.9
The need for the early identification and accurate assessment of these problem behaviors has led to the development of various measures designed to assess behavioral problems in ASD. The most widely used instruments include the Aberrant Behavior Checklist (ABC),10 Behavior Problems Inventory (BPI),11 Nisonger Child Behavior Rating Scales (NCBRF),12 Behavior Assessment System for Children (BASC),13 Home Situations Questionnaire-Pervasive Developmental Disorder version (HSQ-PDD),14 and Child Behavior Checklist (CBCL).15 These instruments vary in scope, length, targeted age and population. The ABC and NCBRF were initially designed to measure problem behaviors in individuals with intellectual disabilities, and the BASC and CBCL are tools that do not target a specific psychiatric diagnosis. Most tools mentioned are broad-band instruments and, as they encompass an extensive range of psychopathology, only a few items target problem behaviors.11 Moreover, evidence on the validity and reliability of these tools for individuals with ASD is limited. A recent review of the properties of tools that measure behavior problems in children with autism reported that, among 12 tools, there was evidence available for only 6, and most showed mixed results.16 The CBCL and HSQ-PDD had the most robust measurement properties. However, the CBCL was developed to be used with typically developing children, and there was no evidence of its construct validity in children with ASD, whereas the HSQ-PDD was developed recently, and the developers are continuing to explore the most effective groupings of items.17
In South Korea, there has been increasing concern about managing the problem behaviors of patients with ASD, and the “Developmental Disabilities Assistance and Bill of Rights Act,” enacted in November 2015, announced a plan to establish nationwide behavior development enhancement centers for the management of problem behavior in patients with developmental disorders. However, despite the increasing social awareness of the strong likelihood of problem behaviors in ASD, there is no instrument in South Korea with well-established validity and reliability that can accurately assess problematic behavior in individuals with ASD. With the exception of the CBCL and BASC, none of the pre-existing instruments has been standardized in Korean,1819 and all were developed in Western countries, thus limiting their widespread use in South Korea.
Kim et al. (unpublished) developed a tool to assess challenging behavior in patients with ASD, the Korean Comprehensive Scale for the Assessment of Challenging Behavior in Developmental Disorders (K-CSCB). We pooled items from tools that were considered valid and reliable, collected information from caregivers and therapists about additional symptoms not included in the pooled items, and allowed a group of psychiatrists, educational professionals, clinical psychologists, and behavior therapists to determine face validity. The purpose of this study was to establish the reliability and validity of the K-CSCB. We recruited patients with ASD from both clinical and community settings to widen the generalizability of our study results to patients with ASD with various characteristics.



Participants with ASD, aged 5-22 years, were recruited from hospitals and special education schools during the period of February 2016 to July 2016. The clinical group was recruited from the Seoul National University Children's Hospital and the Seoul National University Bundang Hospital. Diagnoses of ASD were confirmed by board-certified child and adolescent psychiatrists according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV). Individuals with ASD who were receiving special education were recruited from six special education schools in Seoul and Incheon. Exclusion criteria were as follows: 1) schizophrenia or any other childhood-onset psychotic disorder, 2) mood disorders, 3) conduct disorder or oppositional defiant disorder, and 4) inability of the parents to understand or cooperate. The control group was defined as community-dwelling children who were free of any psychiatric diagnoses, and the exclusion criteria were the same as those applied to the ASD group, with the addition of an ASD diagnosis. We only included patients who completed more than 95% (84 out of 89 items) of the K-CSCB.
After a detailed explanation of the study, written informed consent was obtained from the participants' legal guardians, and the study protocol was approved by the Institutional Review Board of Seoul National University Hospital (1406-016-585).


The Korean Comprehensive Scale for assessment of Challenging Behavior in Developmental Disorders (K-CSCB)

This scale was originally developed by Kim et al. (unpublished) to assess problematic behavior during the past 2 months in individuals with developmental disorders, particularly ASD. The items of this scale are listed in Supplementary Table 1 (in the online-only Data Supplement). It initially consisted of 89 items and six subcategories. There were 13 items addressing self-harm, 17 addressing aggressive behavior, 22 addressing stereotyped behavior, 13 addressing unresponsiveness and negativism, 12 addressing attention deficits and hyperactivity, and 12 addressing inappropriate words and behavior. It investigates various behaviors that may appear during the childhood, adolescence, and early adulthood of patients with developmental disorders and was designed to be completed by the primary caretaker of the patient (e.g., parent, teacher, or behavioral therapist). Each item assesses frequency on a five-point scale (from 0 to 4, with 4 being the most frequent) and severity on a four-point scale (from 0 to 3, with 3 being the most severe).

Behavior Problems Inventory

The Behavior Problems Inventory (BPI) is a 52-item narrow-band behavior rating scale for behavior problems in individuals with developmental disorders.11 It assesses the severity and frequency of problem behaviors and includes 14 items related to self-injurious behaviors, 24 items related to stereotypic behaviors, and 11 items related to aggressive/destructive behaviors. The reliability and validity of the BPI have been well established, with a full-scale Cronbach's α of 0.83, test-retest reliability of 0.76, inter-rater reliability of 0.92, and fair criterion validity as assessed by diagnoses of pervasive developmental disorder (PDD) or stereotyped movement disorders.

Child Behavior Checklist 1.5-5

The Child Behavior Checklist (CBCL) 1.5-5 is a 100-item parent-report measure that is widely used to screen for problematic behaviors; however, this instrument does not target any specific psychiatric diagnosis.20 The CBCL 1.5-5 is known to have good reliability and validity.20 The CBCL 1.5-5 is known to have good reliability and validity.2021 The raw scores on seven syndrome scales and five DSM-oriented scales are converted to normalized T-scores (mean=50, standard deviation=10); we used scores on the internalizing, externalizing, aggressive behaviors, and total problems sections to evaluate the concurrent validity of the K-CSCB. The parents of school-aged children with ASD (aged 6-12 years) completed the CBCL 1.5-5. The CBCL 1.5-5 has been found to reliable in measuring problem behaviors in ASD youth over 5 years old.22

Statistical analyses

The demographic and clinical characteristics of the ASD and control groups were compared using independent t-tests for continuous variables and chi-square tests for categorical variables. In regard of missing data, the subscales scores of individuals that were missing more than 1 item were excluded from the analyses and only subscale scores that had not missing items were included. The reliability and validity analyses were based on the severity scores of each subscale. Cronbach's α was calculated to evaluate the internal consistency of the scale and each subscale. The test-retest reliability, correlation between subscales, and correlation between the frequency and severity score of each subscale were calculated using Pearson's correlation analyses. Concurrent validity was assessed using Pearson's correlation coefficients for scores on scales widely used for the assessment of problematic behavior, the BPI and CBCL. Discriminant validity was evaluated by comparing scores in the ASD and control groups via independent t-tests. As there was a significant difference between the ASD and control groups in age and gender, we further compared the scores between the two groups using analysis of covariance (ANCOVA), with age and gender as covariates. To test the diagnostic utility and determine the cut-off score that best discriminates the ASD from the control group, we analyzed the receiver-operating-characteristic (ROC) curve and selected the score with the highest sensitivity and specificity.
All statistical analyses were performed using SPSS ver. 22.0 software (IBM Corp., Armonk, NY, USA), and statistical significance was set at a two-tailed p-value<0.05.


The demographic characteristics of the ASD and control groups are shown in Table 1. We recruited 215 patients with ASD; demographic data were lacking for 1, and 25 completed less than 95% of the K-CSCB. Of the 214 controls recruited, basic demographic data were lacking for 5, and 41 completed less than half the K-CSCB. In total, data from 189 patients with ASD and 168 controls were included in the analyses. The mean age was significantly higher and there were more male participants in the ASD than in the control group. Among the patients with ASD, 89.3% had a nationally registered disability grade, and 55.1% had the highest grade (grade 1), whereas only 1.1% had the lowest grade (grade 3).
The local prevalence of each item in the ASD population is presented in Table 2. All patients with ASD reported at least one problem behavior, and the subscale showing the highest prevalence was “inappropriate words or behavior” (96.4%, ranging from 13.2% to 68.3%), followed by attention deficit/hyperactivity (94.3%) and stereotypy (94.2%). The item with the highest prevalence was “short attention span” (84.7%), and the item with the lowest prevalence was “pulling body parts (eyeballs, tongue, genitalia, etc.)” (3.2%).
The internal consistency of the items was calculated, and six items from the self-harm category and eight items from the aggressive behavior category were deleted due to low item-total correlation (less than 0.3). As a result, the full-scale Cronbach's α was 0.97, and all subscales showed fair internal consistency, ranging from 0.73 to 0.93 (Table 3). We found significant correlations between the frequency and the severity scores of each subscale. Thirty-two participants completed the retest after an interval of 4 weeks, and all subscales except that addressing self-harm showed statistically significant test-retest reliability (p<0.05) (Table 3). There were significant correlations between each pair of subscales (p<0.01) (Table 4).
The K-CSCB total and subscale scores were significantly higher in the ASD than in the control group (p<0.001) (Table 5). Even after including age and gender as covariates, all results remained significant (p<0.001).
The corresponding subscales of the K-CSCB were significantly correlated with the BPI scales addressing self-injury, stereotypic behaviors, and aggressive/destructive behaviors, and the total K-CSCB score was correlated with the total BPI score (Table 6). All K-CSCB subscales, except inappropriate words and behaviors and unresponsiveness, were significantly correlated with the corresponding CBCL subscales for internalizing, externalizing, aggressive behavior, and total behavior scores. The unresponsiveness scores showed significant correlation with internalizing and total CBCL scores.
According to the ROC analysis, the diagnostic utility of the full instrument was significant at 97.7% (p<0.001), and the diagnostic utility of each subscale was also significant (Table 7). The cut-off score of the full instrument with the highest sensitivity and specificity was 12.5 points. At this score, the sensitivity was 90.4%, the specificity was 94.7%, the positive-predictive value was 93.4%, and the negative-predictive value was 92.2%.


The K-CSCB is the first tool developed in Korean designed to evaluate behavioral problems in ASD. It is the only instrument that includes such important items as “repeatedly opens and closes zippers or buttons on clothes,” “repeatedly pursues a particular tactile sensation (e.g., stickers, toothpaste, bubble wrap),” and “overly sensitive to auditory sensations (e.g., echoing sound of auditorium, firecrackers, speaker noise).” All of these items showed a high local prevalence in the ASD population (34.4, 28.6, and 62.0%, respectively). Another strength of this study is that we recruited participants from both clinics and special education schools. As interventions for problematic behavior can be applied not only in the clinical setting (e.g., pharmacotherapy) but also in the school setting (e.g., behavioral therapy), it is important that this instrument was validated in populations from both settings.
This study provides evidence that the K-CSCB is valid and reliable. It showed excellent internal consistency (Cronbach's α=0.97), which exceeded those of the previous aforementioned instruments (BPI=0.83, ABC=0.68-0.90, HSQ-PDD=0.80 and 0.90, CBCL 1.5-50.93).11142123 The test-retest reliability was significant for most subscales; the only exceptions, the self-harm subscale, may be attributable to the small number of participants who completed the retest, causing increased probability of type II errors. Moreover, as self-harm behavior requires more immediate intervention compared to other behaviors, the stability of this construct could vary over time. Further studies with a larger sample size are needed to confirm the test-retest reliability of the K-CSCB. The K-CSCB also showed good discriminant validity and concurrent validity, especially with the BPI (r=0.77).
Whereas most scales measure only the severity of a problem behavior, the K-CSCB simultaneously measures severity and frequency. This may be useful in clinical situations where the severity and frequency of a problem behavior do not match, as a severe problem may be infrequent or a frequent problem may not be severe. However, as the frequency and severity scales showed a strong correlation with each other, little may be gained from retaining both scales, and it may be possible to eliminate one of them. We suggest the flexible use of both scales according to the purpose of data collection.
Total scores on the K-CSCB range from 0 to 225; in the ASD population in our study, the lowest score was 3 and the highest score was 120. All participants in this group reported more than one problematic behavior, which is a higher rate than found in previous studies. A study using the BPI found that three-quarters of the sample endorsed at least one item; the highest prevalence rate among items was 22.2%,11 whereas the highest in this study was 84.8%. The high prevalence of problem behaviors in this study may have been due to the fact that the ASD population was over-represented by individuals with high severity, as more than half had a disability grade of 1 (highest grade). However, comparison with the results of other studies may be difficult due to differences in study populations, sampling strategies, and study designs. As the prevalence of problematic behavior among patients with ASD has not been studied in South Korea, the K-CSCB could be a valuable tool in future epidemiological research.
Kalb and Loeber reported that 25% to 65% of children aged 2-16 years of age have problems with noncompliance.24 Among children with autism, compliance with instructions may be particularly important because of the many requests to comply with adult directions that are issued during early behavioral interventions.25 Despite the high rate of noncompliance in patients with ASD, the HSQ-PDD is the only accepted tool that measures behavioral non-compliance in children with ASD.14 Although the items are not as specific as those in the HSQ-PDD, some of the items on the “unresponsiveness/negativism” subscale of the K-CSCB may reflect non-compliance in patients with ASD (e.g., no response to verbal instruction, no response to structured activities). As the BPI and CBCL do not measure noncompliance per se, further research investigating concurrent validity with HSQ-PDD are needed.
ASD is associated with high rates of comorbidity with attention-deficit/hyperactivity disorder (ADHD), and 30-80% of those with ASD meet the criteria for ADHD.25 Owing to the overlapping characteristics and etiological factors, Rommelse has convincingly argued that the two disorders should be systematically studied together.2627 There was also a high rate of ADHD traits in the ASD population in this study. The most prevalent item was “short attention span,” and 94.3% of the children with ASD had affirmative responses to at least one item in the “attention deficit/hyperactivity” subscale. Thus, we propose that the K-CSCB may have a promising role in future research involving comorbid ASD and ADHD.
There were some noteworthy limitations to this study. First, although the patients with ASD recruited by hospitals were diagnosed by child and adolescent psychiatrists, we relied on parent reports for confirmation of the ASD diagnosis of the participants recruited by special education schools. There was also a significant difference between the ASD and control groups in age and gender. We tried to compensate for this by adding age and gender as covariates, and this did not alter the significance of the results. We used the CBCL 1.5-5 version in school-aged ASD children. However, as the developmental age of ASD children are lower compared to typically developed children, the use of the CBCL 1.5-5 in ASD may capture the behavioral problems in school-age ASD individuals well. Finally, the participants consisted primarily of children and adolescents, which limits the generalizability of our findings to adults with ASD.
The K-CSCB is a new tool for the assessment of problem behavior in patients with ASD, and this study provides evidence for its validity and reliability. As this is the first tool in Korean, we expect that this tool will be widely used in clinical and research settings in South Korea.


This work was supported by a grant of the Korean Social Service R&D Project, Ministry of Health and Welfare, Republic of Korea (HI14C1098).

Supplementary Materials

The online-only Data Supplement is available with this article at

Supplementary Table 1



1. Baird G, Simonoff E, Pickles A, Chandler S, Loucas T, Meldrum D, et al. Prevalence of disorders of the autism spectrum in a population cohort of children in South Thames: the Special Needs and Autism Project (SNAP). Lancet 2006;368:210-215. PMID: 16844490.
crossref pmid
2. Fombonne E. The epidemiology of autism: a review. Psychol Med 1999;29:769-786. PMID: 10473304.
crossref pmid
3. Fombonne E. Epidemiology of pervasive developmental disorders. Pediatr Res 2009;65:591-598. PMID: 19218885.
crossref pmid
4. Kim YS, Leventhal BL, Koh YJ, Fombonne E, Laska E, Lim EC, et al. Prevalence of autism spectrum disorders in a total population sample. Am J Psychiatry 2011;168:904-912. PMID: 21558103.
crossref pmid
5. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th Ed. Washington, DC: American Psychiatric Association; 2013.

6. Horner RH, Carr EG, Strain PS, Todd AW, Reed HK. Problem behavior interventions for young children with autism: a research synthesis. J Autism Dev Disord 2002;32:423-446. PMID: 12463518.
crossref pmid
7. Luiselli JK, Blew P, Keane J, Thibadeau S, Holzman T. Pharmacotherapy for severe aggression in a child with autism: “open label” evaluation of multiple medications on response frequency and intensity of behavioral intervention. J Behav Ther Exp Psychiatry 2000;31:219-230. PMID: 11494958.
crossref pmid
8. Hartley SL, Sikora DM, McCoy R. Prevalence and risk factors of maladaptive behaviour in young children with Autistic Disorder. J Intellect Disabil Res 2008;52:819-829. PMID: 18444989.
crossref pmid pmc
9. Gadow KD, DeVincent CJ, Pomeroy J, Azizian A. Psychiatric symptoms in preschool children with PDD and clinic and comparison samples. J Autism Dev Disord 2004;34:379-393. PMID: 15449514.
crossref pmid
10. Aman MG, Singh N. Aberrant Behavior Checklist Manual. East Aurora, New York: Slosson Educational Publications; 1986.

11. Rojahn J, Matson JL, Lott D, Esbensen AJ, Smalls Y. The Behavior Problems Inventory: an instrument for the assessment of self-injury, stereotyped behavior, and aggression/destruction in individuals with developmental disabilities. J Autism Dev Disord 2001;31:577-588. PMID: 11814269.
crossref pmid
12. Aman MG, Tasse MJ, Rojahn J, Hammer D. The Nisonger CBRF: a child behavior rating form for children with developmental disabilities. Res Dev Disabil 1996;17:41-57. PMID: 8750075.
crossref pmid
13. Kamphaus R, Reynolds C. BASC-2 behavioral and emotional screening system manual. Circle Pines, MN: Pearson; 2007.

14. Chowdhury M, Aman MG, Scahill L, Swiezy N, Arnold LE, Lecavalier L, et al. The Home Situations Questionnaire-PDD version: factor structure and psychometric properties. J Intellect Disabil Res 2010;54:281-291. PMID: 20377705.
crossref pmid
15. Achenback TM, Rescorla L. Manual for ASEBA preschool forms and profiles. Burlington, Bermont: Research Center for Children, Youth and Families, University of Vermont; 2000.

16. Hanratty J, Livingstone N, Robalino S, Terwee CB, Glod M, Oono IP, et al. Systematic review of the measurement properties of tools used to measure behaviour problems in young children with autism. PLoS One 2015;10:e0144649. PMID: 26659821.
crossref pmid pmc
17. Chowdhury M, Aman MG, Lecavalier L, Smith T, Johnson C, Swiezy N, et al. Factor structure and psychometric properties of the revised Home Situations Questionnaire for autism spectrum disorder: The Home Situations Questionnaire-Autism Spectrum Disorder. Autism 2016;20:528-537. PMID: 26187059.
crossref pmid
18. Oh KJ, Lee H, Hong KE, Ha EH. K-CBCL. Seoul: Chung Ang Aptitute Publishing Co; 1997.

19. Ahn CM, Ebesutani C, Kamphaus R. A psychometric analysis and standardization of the Behavior Assessment System for Children-2, Self-Report of Personality, Child Version among a Korean sample. Sch Psychol Q 2014;29:198-212. PMID: 24015981.
crossref pmid pdf
20. Achenback TM, Rescorla L. Manual for the ASEBA Preschool Forms and Profiles: A Integrated System of Multi-Informant Assessment. Burlington, VT: University of Vermont, Deparment of Psychiatry; 2001.

21. Pandolfi V, Magyar CI, Dill CA. Confirmatory factor analysis of the Child Behavior Checklist 1.5-5 in a sample of children with autism spectrum disorders. J Autism Dev Disord 2009;39:986-995. PMID: 19263208.
crossref pmid pmc
22. Basten M, van der Ende J, Tiemeier H, Althoff RR, Jolien R, Jaddoe VW, et al. Nonverbal intelligence in young children with dysregulation: the Generation R Study. Eur Child Adolesc Psychiatry 2014;23:1061-1070. PMID: 24802760.
crossref pmid
23. Karabekiroglu K, Aman MG. Validity of the aberrant behavior checklist in a clinical sample of toddlers. Child Psychiatry Hum Dev 2009;40:99-110. PMID: 18600444.
crossref pmid
24. Kalb LM, Loeber R. Child disobedience and noncompliance: a review. Pediatrics 2003;111:641-652. PMID: 12612249.
crossref pmid
25. Fischetti AT, Wilder DA, Myers K, Leon-Enriquez Y, Sinn S, Rodriguez R. An evaluation of evidence-based interventions to increase compliance among children with autism. J Appl Behav Anal 2012;45:859-863. PMID: 23322942.
crossref pmid pmc
26. Rommelse NN, Geurts HM, Franke B, Buitelaar JK, Hartman CA. A review on cognitive and brain endophenotypes that may be common in autism spectrum disorder and attention-deficit/hyperactivity disorder and facilitate the search for pleiotropic genes. Neurosci Biobehav Rev 2011;35:1363-1396. PMID: 21382410.
crossref pmid
27. Rommelse NN, Altink ME, Fliers EA, Martin NC, Buschgens CJ, Hartman CA, et al. Comorbid problems in ADHD: degree of association, shared endophenotypes, and formation of distinct subtypes. Implications for a future DSM. J Abnorm Child Psychol 2009;37:793-804. PMID: 19308723.
crossref pmid pmc
Table 1

Demographic characteristics of participants


ASD: autism spectrum disorder, SD: standard deviation

Table 2

Local prevalence of each item: ASD group


ASD: autism spectrum disorder

Table 3

Internal consistency, test-retest reliability, and frequency-severity correlation of K-CSCB subscales


*p<0.05, **p<0.01. K-CSCB: Korean Comprehensive Scale for the Assessment of Challenging Behavior in Developmental Disorders, F-S: frequency-severity correlation

Table 4

Correlation between subscales of the K-CSCB


*p<0.01. K-CSCB: Korean Comprehensive Scale for the Assessment of Challenging Behavior in Developmental Disorders

Table 5

Comparison of the K-CSCB subscales score of ASD and control participants


K-CSCB: Korean Comprehensive Scale for the Assessment of Challenging Behavior in Developmental Disorders, ASD: autism spectrum disorder

Table 6

Correlation of K-CSCB subscales with BPI and CBCL subscales


*p<0.05, **p<0.01. K-CSCB: Korean Comprehensive Scale for the Assessment of Challenging Behavior in Developmental Disorders, BPI: Behavior Problems Inventory, CBCL: Child Behavior Checklist

Table 7

Area under ROC curve for each subscale


*p<0.001. ROC: receiver operating characteristic, AUC: area under curve, SE: standard error


Browse all articles >

Editorial Office
#522, 27, Seochojungang-ro 24-gil, Seocho-gu, Seoul 06601, Korea
Tel: +82-2-717-0892    E-mail:                

Copyright © 2024 by Korean Neuropsychiatric Association.

Developed in M2PI

Close layer
prev next