This cross-cultural study aims to examine the psychometric properties of the Social Responsiveness Scale (SRS) and to determine the best-estimate-cut-off scores for the diagnosis of autism spectrum disorder (ASD) in Korean preschool children.
A total of 563 children was recruited from multiple sources, including Seoul National University Bundang Hospital. Participants were assessed by a multidisciplinary research team using multiple diagnostic tools based on DSM-5 diagnostic criteria. Discriminative validity was tested by comparing the difference in SRS scores between ASD and non-ASD groups. Convergent validity was tested by examining correlations between SRS scores with other diagnostic instruments. A receiver operation characteristic curve analysis was conducted to test the sensitivity and specificity of SRS and to determine the best-estimate-cut-off scores for screening ASD in Korean preschool children.
There were significant differences in the total SRS scores between the ASD (n=242) and non-ASD group (n=321) (p<0.01, 95% confidence interval [CI]: 14.08–17.24). The differences were significant even after adjusting for age. SRS scores were significantly correlated with other prescreening diagnostic tools for ASD. The best-estimate cut-off score to screen ASD was 55 (area under curve=0.88, sensitivity 78.1%, specificity 86.6%).
The SRS is a valid and reliable instrument to screen and aid in the diagnosis of ASD in Korean preschool children. The adjusted cut-off scores, notably lower than in the original U.S. version, may yield a more accurate diagnosis by reflecting transcultural differences.
Research places emphasis on the early diagnosis of the autism spectrum disorder (ASD) as the literature consistently highlights the critical role of early intervention in achieving a more positive prognosis [
By examining the traits highly distinct to ASD, the SRS quantitatively measures the continuum of autism symptom severity [
As this screening instrument was originally developed in USA and initially validated with the US population [
In Korea, to the best of authors’ knowledge, there exists only a single validation study of the Korean version of SRS (K-SRS) focusing on children aged 6 to 12 years old, which showed that the K-SRS has an adequate degree of reliability and validity among Korean participants [
Furthermore, overestimating ASD traits may potentially result in the wastage of diagnostic/therapeutic resources, while underestimating may result in the failure to make an early diagnosis; both are possible depending on the cut-off score. Literature has repeatedly pointed out that cultural differences may attenuate the original statistical power of diagnostic tools for ASD, especially for Asian populations [
Given this context, this study aims to examine the psychometric properties of the Korean version of the SRS-2 Preschool form, including the questionnaire’s discriminative validity against ASD, other developmental disorders (OD), and the typical development (TD) group among Korean preschool children. Additionally, we determined the best-estimate-cut-off scores for the screening of ASD in Korean preschool children while also conducting a satellite analysis to examine the correlation between the SRS scores of children with ASD and their parents and perform a preliminary exploration on the inheritable end phenotype of ASD, which is the main target of SRS evaluation.
The participants for this study were drawn from the databases of two previous and ongoing studies, including a genetic study regarding ASD’s biomarkers and a study on the development of an early ASD screening instrument [
ADOS and ADOS-2 are semi-structured standard diagnostic tools employing a play-based approach to detect the ASD symptoms. ADOS consisted of modules 1 to 4, while ADOS-2 consisted of module T, and these aimed to evaluate both verbal and nonverbal communication and social interaction as well as the child’s playing and imagination. Both modules targeted children below the age of 30 months. The examined child is assessed on different domains related to key symptoms of ASD: communication, social interaction, combined communication and interaction, play and imagination, and RRBs using ADOS; social interaction and communication, RRBs, and total and comparison scores using ADOS-2. Children are classified into three groups based on their final results: autism, ASD, and non-spectrum combined. Participants were assessed using ADOS until ADOS-2 was published in Korea; from then on, ADOS-2 was used. Based on the ADOS-2 algorithms, premeasured ADOS scores were rescored into an ADOS-2 score form. Western Psychological Services, the creator of ADOS-2, approved the Korean translation of ADOS-2 [
ADI-R is a semi-structured parent interview tool for parents with children aged 2 or older. Generally, it supplements the direct observation of children using the ADOS assessment. The contents of the interview consist of early development, acquisition and loss of language/other functions, language and communication skills, social development and play, interest and behavior, and general behavior. It evaluates three maladjustment symptoms of ASD: a persistent and prominent deficit in social interaction/communication, RRBs, and restricted and repetitive interests, based on diagnostic algorithms of DSM-IV and the International Statistical Classification of Diseases and Related Health Revisions 10th Revision [
K-CARS detects and measures the severity of pervasive developmental disorder symptoms based on both the parents’ report and direct observation of children. K-CARS consists of 15 items evaluating different domains including socialization, language and nonverbal communication, restricted emotional responses or unusual behaviors and interests, and sensory sensitivities. Each item is scored from 1 (no impairment) to 4 (severe impairment). Upon the completion of the assessment, the child is categorized as having either mild, moderate, or severe symptoms of autism. We used a cut-off score of 24 to diagnose ASD based on the previous standardization process [
Based on the ADI-R, the Social Communication Questionnaire (SCQ) is a screening tool consisting of 40 questions that ask parents or caregivers about ASD-related symptoms (communication, mutual social interaction, RRBs and interests). There are two types: “Lifetime Form” and “Current Form.” Lifetime Form is based on the overall development of the child and Current Form is based on the child’s behavior in the last three months. Each question can be answered with a “yes” or “no,” while the first question checks the child’s ability to speak more than two words. If the first question is answered with “Yes,” the total score is the sum of the scores of questions 2 to 40. If the first question is answered with “No,” the total score is the sum of the scores of questions 8 to 40. We used a cut-off score of 10 to diagnose a high risk of ASD based on a standardization study previously conducted in Korea [
With the approval of the original publisher, Western Psychological Services, we translated and back-translated the SRS-2 preschool form and received permission to use it for the study. The SRS preschool form consists of 65 questions that ask parents or caregivers, including teachers, about the characteristics of the child’s social interactions based on their behavior for the past six months. Each question is answered on a scale ranging from “Not at all” (0 points) to “Almost always” (3 points), with a total score of 195. The higher the score, the lower the social dexterity.
The original SRS-2 preschool version was intended for children above the age of 30 months. However, as a substantial number of participants aged less than 30 months were recruited, we planned to conduct a subgroup analysis for children over 30 months and for those below 30 months to determine the validity of SRS in the age group below the original stipulation and to prevent potential bias from lenient age inclusion. In this study, the SRS-2 was filled out by a primary caregiver who was familiar with the developmental process of the child. The SRS’s testretest reliability coefficient of the U.S. preschool version was 0.75 [
The SRS-2 adult self-report form (the SRS-2 ASR) is indicated for examinees aged 19 through 89. It consists of 65 questions scored on a Likert scale identical to the preschool form, with each item being scored from 0–3 points.
The VABS-2 was designed to assess the adaptive behavior and individual and social skills necessary for independent daily living over a lifetime, and is conducted through a survey interview and evaluation from parents and caregivers, including teachers. It aims to assess the child’s communication, daily living skills, social skills, motor skills, as well as maladaptive behaviors which can be both internal and external. Items are rated from 0 to 2 with 0 indicating that a certain skill is not used by the examinee at all; 1 indicating that the skill is used infrequently; and 2 indicating that the skill is used frequently. VABS-2 can also be used to screen the service eligibility, to plan additional interventions, and to track and review the changes in children with ASD post-intervention.
SELSI evaluates the receptive language development and expressive language for children between the ages of 5 to 36 months. Consisting of 56 questions, SELSI provides the developmental age, language development in terms of percentile and standard deviation, and the approximate level of development for children with delays.
Recruited children were assessed using the aforementioned diagnostic instruments. Parents were evaluated using the SRS-2 adult self-report form. Basic demographics were obtained from all participants. Developmental history and family history were extensively surveyed for diagnostic aid. Qualified diagnostic assessment staff, education professionals specializing in ASD evaluation, and experienced clinicians, including two licensed child psychiatrists, made the final clinical best estimate diagnoses. Any uncertain results were discussed using a multidisciplinary team approach at weekly meetings, where the videotaped evaluations were discussed and reviewed to draw conclusions. We classified the children into three diagnostic groups; the participant group with ASD included children with a final clinical best-estimate diagnosis of ASD based on all their exam results and information, following the DSM-5 criteria. As for children aged 18–23 months, while this age is sometimes considered too young for confirming a diagnosis, we diagnosed the child with ASD if they were assessed to have “mild-to-moderate concerns” or “moderate-to-severe concerns” in module T of ADOS-2, and when their parents reported concurrent and equivalent degrees of concerns in ADI-R. The OD group included children whose levels of development were below -1 standard deviation in SELSI, or those who scored less than 70 points in any domain of VABS. The TD group included children showing a normal range of development, and those who did not meet the criteria for the diagnosis of either ASD or TD.
The discriminative validity of SRS was tested by conducting a t-test and analysis of covariance to compare differences in scores between ASD and non-ASD groups. The convergent validity of SRS with other diagnostic instruments including K-CARS and K-SCQ was examined using Pearson’s correlation. Finally, we conducted a receiver operation characteristic (ROC) curve analysis to test the sensitivity and specificity of SRS, and to determine the best-estimate-cut-off scores for screening ASD in Korean preschool children. An area under curve (AUC) serves as the indicator of performance, and a value of 0.50 means that the diagnostic tool has no discriminative power as compared to the gold standard, while a value of 1.0 means complete power. A value of 0.70 or higher indicates an appropriate level of discriminative accuracy. Sensitivity, specificity, positive predictive values, and negative predictive values were calculated. We used MedCalc for Windows, version 20 (MedCalc Software, Ostend, Belgium) for a ROC curve analysis and SPSS statistics 25 and Subscription (IBM Corp., Armonk, NY, USA) for other analyses. The confidence interval was set at 95%.
A total of 563 children ranging from 10 to 65 months of age participated in the study. The average age was 31.47 months (SD=9.95), and the majority of participants were male (n=379, 67.32%). A total of 335 children aged 30 months and above were recruited (n=335, mean age=38.16, SD=6.14), while 228 children aged below 30 months were recruited (n=228, mean age=21.67, SD=5.16).
Participants were classified according to the clinical best estimate diagnosis into three groups: ASD (n=242), OD (n=74), and TD (n=247). Upon combining the OD and TD into the non-ASD group, there were 321 participants. The mean age of the ASD group was 35.64 months (SD=9.27), higher than the OD group (mean age=29.66, SD=8.03) and TD group (mean age=27.93, SD=9.6). The mean VABS score was lowest in the ASD group (mean=69.28, SD=14.29), followed by the OD group (mean=81.26, SD=11.74), and the TD group (mean=101.04, SD=12.80). K-CARS scores, on the other hand, were lowest in the TD group (mean=16.65, SD=1.94), and higher in the OD group (mean=20.02, SD=4.37) and the ASD group (mean=31.44, SD=5.08), displaying a similar pattern with their SCQ scores (TD mean=6.20, SD=4.03; OD mean=8.01, SD=4.58; ASD mean=15.47, SD=6.38).
The demographic characteristics of the participants, along with the participants’ assessment scores other than SRS, are summarized in
Overall, the mean SRS score of the ASD group was 63.26 with an SD of 10.73, whereas the mean score of the non-ASD group was 47.60, with an SD of 7.46, indicating a statistically significant difference (t=19.44, p<0.01). Statistical significance was maintained after adjusting for age (F=362.84, p<0.01). When we restricted the participants according to the original indication age of SRS preschool form (children aged 30 months and above), the mean score of the ASD group was 64.39 (SD=10.12) and the mean score of the non-ASD group was 46.75 (SD=8.28), still showing a statistical difference (t=17.55, p<0.01) with and without adjusting for age (F=289.21, p<0.01). As for participants below the age of 30 months, there was still a significant difference between the ASD and non-ASD groups, with and without adjusting for age (
Upon classifying all the participants into three groups (ASD, OD, and TD), the mean score of the OD group was 52.01 (SD=8.07) and the mean score of the TD group was 46.28 (SD=6.74), with a statistically significant difference among all three groups (F=147.05, p<0.01). Among the three groups, the score was highest for the ASD group, followed by OD, and lastly, the TD group. The significance was maintained after adjusting for age (F=147.05, p<0.01). In the subgroup analysis of children aged above 30 months, the mean score of OD was 50.98 (SD=8.82) and the mean score of TD was 45.22 (SD=7.54), and maintained a significant difference (F=158.16, p<0.01) within the three groups, even after adjusting for age (F=132.67, p<0.01). Again, there was a significant difference among the three groups in terms of SRS mean scores for children below the age of 30 months, with and without the age adjustments (
We analyzed the correlations between SRS, SCQ and K-CARS. There was a significant correlation between SRS scores and SCQ scores (r=0.800, p<0.01) and between SRS scores and K-CARS scores (r=0.730, p<0.01) for all the participants. The positive correlation was maintained after adjusting for age (r=0.802, p<0.01 in SRS & SCQ; r=0.711, p<0.01 in SRS & K-CARS). When we restricted the analysis to include only children above the age of 30 months, a stronger correlation was established between the measures (r=0.893, p<0.01 in SRS & SCQ; r=0.737, p<0.01 in SRS & K-CARS). The correlation became weaker but still remained significant when we restricted the analysis to include only children below the age of 30 months (r=0.695, p<0.01 in SRS & SCQ; r=0.657, p<0.01 in SRS & K-CARS).
Upon analyzing the correlation between the SRS scores of all the children and those of their parents, there was a weak correlation between the scores of children and their mothers (r=0.175, p<0.01). There was no significant correlation between the SRS scores of children above 30 months old and those of their parents. Still, there was a weak significant correlation between the SRS scores of children below 30 months old and their mothers (r=0.236, p<0.05). There were no significant correlations between the SRS scores of children and that of their fathers in both age groups. There was no significant correlation between the SRS scores in children diagnosed with ASD and those of their parents, while the scores of children without ASD had a weak yet statistically significant correlation with those of both of their parents (mother-non-ASD child: r=0.35; fathernon-ASD child: r=0.24; both p<0.01).
We conducted a receiver operating characteristic (ROC) analysis to generate the optimal cut-off score to diagnose ASD in SRS. Upon analysis of the data of all participants, SRS showed solid diagnostic accuracy with an AUC of 0.883. When we apply the cut-off score of 55.50, the test showed optimal results with a sensitivity of 78.1% and specificity of 86.6% (
In group of children aged above 30 months, SRS still maintained its strong diagnostic accuracy with an AUC of 0.904. The optimal cut-off score was 55, with a sensitivity of 82.7% and specificity of 86.0% (
This study is the first to establish the validity of the SRS preschool form when used on Korean children. The mean scores of the SRS were statistically different between the ASD and non-ASD group, and these also significantly differed among the ASD, OD, and TD groups. Adjusting for age did not affect the significance. This may suggest that SRS may be employed not only as a screening instrument to differentiate ASD and non-ASD, but may also aid in the in-depth screening and diagnosis of ASD, supporting the endeavor to detect and differentiate ASD from other development disorders. Another study by the author has also demonstrated the high validity of the Korean ADI-R in ASD detection in comparison with the OD group [
Furthermore, the significant difference in SRS mean scores was maintained in the subgroup analysis of children aged below 30 months. The result supports that the SRS preschool form may sustain its diagnostic efficacy in Korean children population even below its original indicated age group, and can thus aid in prescreening procedures for children below 30 months. Considering the importance of making the earliest possible diagnosis and providing intervention in the course of managing ASD [
Our results correspond with literature based on the East Asian preschool population. Stickley et al. [
We verified the convergent validity of SRS by identifying relevant correlations of SRS with SCQ and K-CARS. There were positive and significant correlations between SRS and SCQ, and SRS and K-CARS across all age groups, but this correlation was strongest among children aged above 30 months, which was the age group for which the SRS preschool form was originally designed.
According to our analysis, the optimal cut-off score of SRS was 55 for the entire age group, which was substantially lower than the original U.S. cut-off score of 60 which is also currently being employed in the Korean clinical field. The findings support the previous study on the use of SRS on the Korean school age population which found much lower optimal cut-off scores, 52 for males and 51 for females, compared to the U.S. cut-off scores of 70 for males and 65 for females [
In addition, as SRS is a diagnostic measure which mainly depends on the self-reports of the main caregivers, the difference in the optimal cut-off score may reflect cultural differences or even differing health literacy levels among the caregivers in each country, featuring a wide spectrum of perspectives in the evaluation and classification of children’s behavior as typical or autistic based on their cultural background. The literature has shown consistent differences between the original and Korean cut-off scores in other ASD diagnostic tools. For example, the original cut-off score of K-CARS was 30, but the optimal cutoff score based on Korean data was 28.5 for all participants (sensitivity=0.871, specificity=0.8), and 24.5 for the high-functioning ASD group (sensitivity=0.759, specificity=0.771) [
The SRS has been employed as a surrogate to examine and quantify the relationship of autistic traits within the family in past studies. Constantino and Todd7 showed statistically significant intraclass correlation (ICC) between the SRS scores of children and their parents (mother–daughter=0.41, mother–son=0.38, father–daughter=0.49, father–son=0.58). We also conducted the satellite analysis exploring the correlation between the SRS scores of children and their parents. There was a significant positive correlation between the SRS scores of children and their mothers overall and in the group aged 30 months and below.
However, when we confined the analysis to children with ASD, their scores had no significant correlation with those of their parents. In contrast, there was a weak but significant correlation between the scores of children without ASD with those of both their parents. These results replicate the pattern observed in previous studies. A study by Lyall et al. [
The absence of a correlation between the SRS scores of children with ASD and those of their parents has been postulated to be due to the restriction in score ranges in children with ASD compared to children without ASD [
The study has several limitations. First, multiple evaluators conducted the assessment procedures resulting in uncertain inter-rater reliability, which may reduce the accuracy and coherent validity of the entire data. Secondly, there was a relatively small number of participants aged below 30 months with ASD, which may further reduce the statistical validity in the relative subgroup analysis. The optimal cut-off scores in the age group below 30 months notably differed from the other two groups, and had the lowest AUC, sensitivity, and specificity. Lastly, the adult sample was relatively small as well, almost half of the child participants, which may further reduce the statistical power of our satellite study.
In spite of the limitations above, this study provided a rational basis to employ the SRS preschool form for screening and diagnosing ASD among the population of Korean children. The SRS preschool form may also sustain its efficacy under the originally intended age group. Furthermore, the study’s adjusted cut-off scores are much lower than the original scores. The application of these customized cut-off scores, which reflect the transcultural difference, would further increase the diagnostic power of the SRS in the course of ASD evaluation.
The datasets generated or analyzed during the study are not publicly available due to the fact that the data are collected from the minors and are subject to IRB review, but are available from the corresponding author on reasonable request.
The authors have no potential conflicts of interest to disclose.
Conceptualization: Jeeyoung Chun, Guiyoung Bong, Hee Jeong Yoo. Data curation: Jeeyoung Chun, Guiyoung Bong, Hee Jeong Yoo. Formal analysis: Jeeyoung Chun, Guiyoung Bong, Hee Jeong Yoo. Funding acquisition: Hee Jeong Yoo. Investigation: Jeeyoung Chun. Methodology: Jeeyoung Chun, Hee Jeong Yoo. Project administration: Hee Jeong Yoo. Resources: Hee Jeong Yoo. Software: Jeeyoung Chun. Supervision: Hee Jeong Yoo. Validation: Hee Jeong Yoo. Visualization: Jeeyoung Chun. Writing—original draft: Jeeyoung Chun. Writing—review & editing: Guiyoung Bong, Jae Hyun Han, Miae Oh, Hee Jeong Yoo.
This work has been supported by the Korea Health Industry Development Institute, Social Service R&D project (HI14C2719), and by the Institute of Information & communications Technology Planning & evaluation (IITP) grant funded by the Korea government (MSIT) (No. 2019-0-00330, Development of AI Technology for Early Screening of Infant/Child Autism Spectrum Disorders based on Cognition of the Psychological Behavior and Response).
The authors thank Division of Statistics in Medical Research Collaborating Center at Seoul National University Bundang Hospital for statistical analyses.
Receiver operating characteristic curve of the SRS for ASD vs. non-ASD. SRS, Social Responsiveness Scale; ASD, autism spectrum disorders; AUC, area under the curve.
ROC Curve of the SRS for ASD vs. non-ASD for children aged 30 months and older. ROC, receiver operating characteristic; SRS, Social Responsiveness Scale; ASD, autism spectrum disorders; AUC, area under the curve.
ROC Curve of the SRS for ASD vs. non-ASD for children aged 30 months. ROC, receiver operating characteristic; SRS, Social Responsiveness Scale; ASD, autism spectrum disorders; AUC, area under the curve.
Demographic result
N | Mean age±SD | VABS mean±SD | K-CARS mean±SD | SCQ mean±SD | |
---|---|---|---|---|---|
Total | 563 | 31.47±9.95 | 84.87±19.94 | 23.35±8.01 | 10.33±6.82 |
Sex | |||||
Male | 379 | 32.43±9.76 | 82.03±19.27 | 24.36±7.95 | 11.18±6.72 |
Female | 184 | 29.51±10.07 | 90.73±20.08 | 21.30±7.77 | 8.64±6.71 |
Age | |||||
≥30 months | 335 | 38.16±6.14 | 79.54±20.18 | 25.36±8.27 | 11.14±7.33 |
<30 months | 228 | 21.67±5.16 | 92.63±16.82 | 20.52±6.70 | 9.18±5.82 |
Diagnosis | |||||
ASD | 242 | 35.64±9.27 | 69.28±14.29 | 31.44±5.08 | 15.47±6.38 |
Non-ASD | 321 | 28.33±9.28 | 96.41±15.09 | 17.44±3.06 | 6.63±4.23 |
Diagnosis in 3 group | |||||
ASD | 242 | 35.64±9.27 | 69.28±14.29 | 31.44±5.08 | 15.47±6.38 |
DD | 74 | 29.66±8.03 | 81.26±11.74 | 20.02±4.37 | 8.01±4.58 |
TD | 247 | 27.93±9.60 | 101.04±12.80 | 16.65±1.94 | 6.20±4.03 |
ASD, autism spectrum disorders; DD, developmental disorder; TD, typical development; SD, standard deviation; VABS, Vineland Adaptive Behavior Scales; K-CARS, The Korean version of Childhood Autism Rating Scale; SCQ, Social Communication Questionnaire
Mean differences in SRS score between ASD, non-ASD (OD+TD)
N | Mean±SD | Standard error of the means | t-value | p | 95% CI | Mean differences adjusted by the age |
||
---|---|---|---|---|---|---|---|---|
F-value | p | |||||||
Total | 563 | 54.33±11.88 | ||||||
Diagnosis | 19.44 | <0.01* | 14.08–17.24 | 362.84 | <0.01* | |||
ASD | 242 | 63.26±10.73 | 0.69 | |||||
Non-ASD | 321 | 47.60±7.46 | 0.42 | |||||
Over 30 months | 17.55 | <0.01* | 15.66–19.62 | 289.21 | <0.01* | |||
ASD | 185 | 64.39±10.12 | 0.74 | |||||
Non-ASD | 150 | 46.75±8.28 | 0.68 | |||||
Below 30 months | 6.8 | <0.01* | 7.93–14.51 | 79.37 | <0.01* | |||
ASD | 57 | 59.56±11.86 | 1.57 | |||||
Non-ASD | 171 | 48.34±6.59 | 0.50 |
Mean differences compared with independent t-test, *p<0.05.
Age adjusted with analysis of covariance, *p<0.05.
ASD, autism spectrum disorders; OD, other developmental disorder; TD, typical development; SD, standard deviation; CI, confidence interval
Mean differences in SRS score between ASD, OD & TD
N | Mean±SD | F | p | Mean differences adjusted by the age |
||
---|---|---|---|---|---|---|
F-value | p | |||||
Total | 563 | 54.33±11.88 | ||||
Diagnosis | 229.08 | <0.01* | 147.05 | <0.01* | ||
ASD | 242 | 63.26±10.73 | ||||
OD | 74 | 52.01±8.07 | ||||
TD | 247 | 46.28±6.74 | ||||
Over 30 months | 158.16 | <0.01* | 132.67 | <0.01* | ||
ASD | 185 | 64.39±10.12 | ||||
OD | 40 | 50.98±8.82 | ||||
TD | 110 | 45.22±7.54 | ||||
Below 30 months | 50.61 | <0.01* | 21.95 | <0.01* | ||
ASD | 57 | 59.56±11.86 | ||||
OD | 34 | 53.24±7.07 | ||||
TD | 137 | 47.12±5.90 |
Mean differences compared with analysis of variance, *p<0.05.
Age adjusted with analysis of covariance, *p<0.05.
ASD, autism spectrum disorders; OD, other developmental disorder; TD, typical development; SD, standard deviation; CI, confidence interval