A Pilot Study on the Validity and Reliability of Korean School Readiness Inventory on Children With Autism Spectrum Disorder

Article information

Psychiatry Investig. 2023;20(10):940-945
Publication date (electronic) : 2023 October 24
doi : https://doi.org/10.30773/pi.2023.0137
Department of Psychiatry, Soonchunhyang University Bucheon Hospital, Soonchunhyang University College of Medicine, Bucheon, Republic of Korea
Correspondence: Jeewon Lee, MD, PhD Department of Psychiatry, Soonchunhyang University Bucheon Hospital, Soonchunhyang University College of Medicine, 170 Jomaru-ro, Wonmi-gu, Bucheon 14584, Republic of Korea Tel: +82-32-621-5063, Fax: +82-32-621-6950, E-mail: jwlee03@schmc.ac.kr
Correspondence: Soyoung Irene Lee, MD, PhD Department of Psychiatry, Soonchunhyang University Bucheon Hospital, Soonchunhyang University College of Medicine, 170 Jomaru-ro, Wonmi-gu, Bucheon 14584, Republic of Korea Tel: +82-32-621-5063, Fax: +82-32-621-6950, E-mail: irenelee@schmc.ac.kr
Received 2023 May 2; Revised 2023 July 6; Accepted 2023 August 2.

Abstract

Objective

Successful transition to school is of great importance to children with autism spectrum disorder (ASD). The purpose of this study was to develop a school readiness inventory for Korean children with ASD, and demonstrate its content validity and reliability.

Methods

The Korean School Readiness Inventory (K-SRI) was developed to assess current levels of some fundamental skills needed for attending school for children with ASD. The K-SRI was comprised of four subscales and 16 test items: Self-help skills, Social and emotional development, School behavior, and Literacy and numeracy skills. For content validity, six experts rated the validity of the test items. Lawshe’s Content Validity Ratio (CVR) was calculated. For reliability, parents of 22 children with ASD entering school completed the KSRI twice. Cronbach’s alpha coefficient was calculated for internal consistency. The test–retest reliability was assessed using the intraclass correlation coefficient (ICC).

Results

All the items except two items in the literacy and numeracy skills did not show a CVR of 1. The two items were deleted resulting in a 14-item inventory. The Cronbach’s alpha coefficient of the K-SRI was 0.93, showing good internal consistency reliability. The test– retest reliability results showed ICC value of 0.93 (p<0.001), which indicates good stability.

Conclusion

A parent-rated, 14-item school readiness inventory for Korean children with ASD were developed and preliminary evidence of its content validity and reliability were demonstrated in this study. The present study provides a basis for future studies that would further help evaluate and promote school readiness of the children with ASD.

INTRODUCTION

Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by deficits in social interaction and communication and repetitive and restricted patterns of behaviors and interests [1]. As the importance of early detection and early intervention for ASD has been highlighted, children as young as 2 to 3 years old are being treated [2]. Early intervention supports children with ASD in all areas of development, including language, cognition, motor skills, social skills, and adaptive functioning. After the period which focuses on early intervention, one of the most important challenges the child faces may be attending school for the first time [3].

Entering school is a significant transition for any child, but it has a greater importance for a child with ASD [3]. Transition to a new physical, social, academic environment could be especially difficult for a child with social and communication deficits. Children with ASD can have more externalizing behaviors and difficulties in self-regulation, resulting in low attendance and participation compared to the typically developing peers [4,5]. They may have higher conflict and lower closeness with the teachers and the peers [6]. They may even become victims or perpetrators of school bullying [7]. Successful transition to school could have a significant impact on the social and cognitive developmental trajectory of a child with ASD [8].

School readiness is a measure of a child’s age-appropriate preparedness and competencies for successful transition to school [9]. According to the National Education Goals Panel, the domains of school readiness of an individual child include 1) health and physical development, 2) emotional well-being, 3) approaches to learning, 4) communicative skills, and 5) cognition and general knowledge [10]. The American Academy of Pediatrics recommends this framework as a basis for assessing preschool-aged children at risk for poor educational outcome [11]. Parents of children with ASD want to know if their child will be able to attend school like other children. Clinicians need to discuss with parents what fundamental skills should be acquired to successfully adapt in school [12]. Also, children with ASD are placed in a variety of educational settings: special education schools, mainstream schools, and specialized support classes in the mainstream schools. However, when deciding the educational setting the child should be placed in, parents or clinicians have no specific guidelines. Previous studies have shown that the placement decision is more influenced by the characteristics of the parents rather than the autism severity or cognitive or language skills of the child [13].

Previous studies that examined school readiness in children with ASD indicated that children with ASD have most difficulties in the areas of social skills and daily living skills.3 Children with ASD and intellectual disability showed significantly poorer understanding of self-social awareness concepts compared to those with only intellectual disability [14]. Children with high-functioning ASD were rated significantly lower in emotion regulation and effortful control than their typically developing peers [5]. Executive function was reported to be the significant predictor of emotional school engagement. Also, children with ASD who demonstrated more advanced theory of mind performance and more positive friendship had higher levels of school readiness [3]. Yet there is a paucity of research on school readiness in children with ASD in Korea. The education system and resources available at school are very different in each country. Therefore, studies on evaluating and promoting school readiness of children with ASD specifically for Korean children are necessary.

Developing an appropriate tool for the assessment of school readiness of children with ASD will be useful in clinical practice and future studies on school readiness of children with ASD. It will help clinicians assess the school readiness of a preschooler with ASD and help the parent make an appropriate school placement decision. Also, examining the school readiness of the child will be helpful in identifying and managing the areas in which the child may need increased support to attend school successfully. The purpose of this study was to develop a school readiness inventory for Korean children with ASD. We tested its content validity and reliability in a representative target population.

METHODS

Study design

The current study consisted of two phases: 1) content validity assessment and 2) reliability assessment. In Phase 1, content validity of the Korean School Readiness Inventory (K-SRI) was assessed through a panel of experts. In Phase 2, the KSRI was administered to children with ASD. Internal consistency and test–retest reliability were evaluated. The study was approved by the Institutional Review Board of Soonchunhyang University Bucheon Hospital (SCHBC 2022-09-015).

Measures

Development of K-SRI

The K-SRI is a proposed tool to screen school readiness skills for children with ASD. We conducted an extensive literature review on many standardized tools on school readiness: Early Development Instrument [15], International Development and Early Learning Assessment [16], Malawi Development Assessment Tool [17], and Brief Early Skills and Support Index [18]. Especially, the two recently proposed tools to assess school readiness scales specifically developed for children with ASD were used as important references [19,20]. The Social Readiness Scale (SRS) targets preschool children with ASD [19]. SRS consists of eight domains: social skills, language skills, attention and memory, approaches towards learning, pre-reading skills, pre-numeracy skills, concept development, and other pre-requisite skills. The Kindergarten Readiness Inventory (K-READI) is developed specifically for nonverbal to minimally verbal children with ASD [20]. K-READI consists of five domains: school activities of daily living, basic school concepts, school-related activities, social participation, and school behavior. One important factor we took into consideration when reviewing other tools was that children begin formal education at age 6 in Korea. Many tools developed in other countries assessed the preschoolers who are at age 5 or younger for school readiness, since formal education started mostly at age 5 [21]. In Korea, beginning “school” refers to entering elementary school, while in many countries it means entering kindergarten.

We developed four subscales and 16 test items for the KSRI: Self-help skills, Social and emotional development, School behavior, and Literacy and numeracy skills. Self-help skills measure the child’s self-care skills for everyday tasks such as dressing, feeding, and toilet training as well as fine motor development skills. Social and emotional development evaluates the child’s ability to socialize and converse with other children. Also, ability to take turns in social activities and ability to tolerate frustration without making a scene. School behavior assesses how the child can sit in the chair, pay attention in class, follow verbal direction from the teacher, and make transitions from one activity to another. Literacy and numeracy skills measure the child’s rough cognitive development and prerequisites for academic learning. The caregiver assesses the child on a scale of 1–4 (1: skills not developed at all, 2: skills not quite developed, 3: skill somewhat developed, and 4: skills very well developed). It takes approximately 10 minutes for the caregiver to assess the child using the K-SRI.

Participants and procedures

Phase 1. content validity assessment

The researchers invited 9 experts to complete a survey to review the content validity of the K-SRI. The experts included 5 child–adolescent psychiatrists, 1 pediatrician, 2 elementary school teachers, and 1 professor from the Department of Special Education. The child–adolescent psychiatrists were all assistant or associate professors from the Department of Psychiatry who had 7 or more years of clinical experience with children with ASD. The pediatrician was an assistant professor from the Department of Pediatrics who specialized in neurodevelopment. The experts rated the validity of the test items in the K-SRI on a scale of 1–4 (1: not relevant, 2: somewhat relevant, 3: quite relevant, and 4: highly relevant). Items were considered “essential” if the expert rated the item 3 (quite relevant) or 4 (highly relevant). Also, an open-ended question for feedback and comments about the items were received from the experts.

Phase 2. reliability assessment

A representative target population of 22 children with ASD were enrolled in the study. The participants were 6 years old, entering elementary school in the year 2023. They were diagnosed with ASD after a thorough neuropsychological assessment including Autism Diagnostic Observational Schedule Second Edition and Autism Diagnostic Interview-Revised [22,23]. The demographic and clinical characteristics of the participants are shown in Table 1. Informed consents were obtained by all parents. The parents completed the K-SRI twice, with at least a 4-week interval. Internal consistency and test–retest reliability were assessed.

Demographic and clinical characteristics of the participants

Statistical analysis

For content validity, Lawshe’s Content Validity Ratio (CVR) was calculated for each item in the inventory [24]. The formula for CVR is as following:

CVR=neN/2N/2,
ne is the number of experts who indicated the item as “essential,” and N is the total number of experts on the panel. CVR values can range between -1 (perfect disagreement) and +1 (perfect agreement), with CVR values above 0 indicating that over 50% of panel members agreed the item as “essential.” As a CVR value gets closer to 1.0, the item is considered more “essential.” For 9 raters, items with a CVR value of 0.778 or above is considered beyond chance agreement (p<0.05, one tailed) [24].

For internal consistency, the Cronbach’s alpha coefficient was calculated [25]. Cronbach’s alpha coefficient can range between 0 and 1. A high value of Cronbach’s alpha coefficient indicates a high level of reliability. The test–retest reliability of the K-SRI was assessed using the intraclass correlation coefficient (ICC) [26]. An ICC value of 0.4–0.75 is considered acceptable, while >0.75 is considered good.

RESULTS

Content validity

The content validity of the K-SRI is demonstrated in Table 2. The experts indicated that most of the items were essential. All the items in the self-help skills, social and emotional development, and school behavior domain showed CVR of 1. The experts’ opinion differed on the 4 items of the literacy and numeracy and skills. Especially, some experts thought questions on writing letters and writing numbers were minimally relevant to the evaluation of school readiness in children with ASD. As a result, we excluded the two items “number writing” and “letter writing” from the inventory.

Content validity of the K-SRI

Reliability

The Cronbach’s alpha coefficient of the K-SRI was 0.93, showing good internal consistency reliability. The values of Cronbach’s alpha coefficients of the subscales ranged from to 0.79 to 0.94, indicating that the scale had good homogeneity (Table 3). The test–retest reliability results showed ICC value of 0.92 (95% confidence interval=0.85–0.96, p<0.001), which indicates good stability. The correlation coefficient for the subscales ranged from 0.79 to 0.93 (Table 3).

Reliability of the K-SRI

DISCUSSION

The K-SRI is a parent-rated instrument developed to assess current levels of some fundamental skills needed for attending school for children with ASD. Items were derived from a review of existing measures and research findings from many other countries. The results of the present study showed that the K-SRI was a valid and reliable instrument for the assessment of school readiness in children with ASD. For content validity, among the 16 test items, 14 items showed CVR value of 0.78 or above. Two items showed CVR of 0.11 and -0.33 and were both removed, resulting in 14 items. Reliability was tested in two ways. First, the Cronbach’s alpha coefficient was 0.93, indicating a high degree of internal consistency. The 4-week test–retest reliability was also examined for consistency of repeated measures. The correlation coefficient of 0.93 confirmed the good stability of the tool.

In the assessment of content validity of the K-SRI, the experts judged the majority of the items to be relevant. Some of the experts expressed concerns about the items in the literacy and numeracy skills domains. They insisted that the ability to read and write, as well as the ability to identify and write numbers were not a prerequisite for entering elementary school. Therefore, the items assessing writing letters and writing numbers were excluded. Nonetheless, we decided to keep the items assessing reading letters and counting numbers. The reason why we did not delete the two items is because even though it may not be ideal, in Korea, it is the reality that most, if not all, children are taught to learn to read and write, as well as know how to do addition/subtraction before entering school. Accordingly, if a child entering school does not know how to read and write and know how to count at all, the child is in need of special attention from the teacher. Not all children have to be proficient in reading, writing, and math, but they do need to have some prerequisites to be academically prepared to start school without being placed in specialized classes. Therefore, we kept the items “reading letters” and “counting numbers.” We decided the items are informative when deciding whether the child needs to be placed in specialized classes or not. We did revise the sentences by adding details to lower the standards: “reading letters without final consonants” and “counting numbers up to 10.”

The limitation of the current study was that the sample size was not large enough to compute a confirmatory factor analysis [27]. A confirmatory factor analysis would have confirmed the construct validity of the K-SRI. Although preliminary steps have been taken to validate the K-SRI in the present study, further studies should be conducted to establish the validity and clinical utility of the K-SRI. We plan to enroll more participants and conduct a long-term prospective study on the participants to evaluate the predictive capability of the KSRI, as well as the clinical factors that most contribute to the successful school adjustment of the children with ASD. We also plan to examine the benefits and drawbacks of placing a child with ASD in either the mainstream or special education settings. The findings may give more guidance to the parents when making a placement decision.

The K-SRI would not only be useful in the assessment of school readiness of the children with ASD, but it could also be used in training the children with ASD to be more ready for inclusive education. Moreover, it could provide the teachers with information on the adaptive skills of the children. The teachers should be prepared to include the children in the educational setting as well. There is a significant lack of schoolbased services for children with ASD in Korea [28]. There is shortage of special education schools for ASD, and not much specialized support is provided in mainstream schools. The concept of school readiness is not only confined to the individual competency of the child. The school readiness also comprises the school’s readiness to provide an optimal learning environment for the child, and family and community support contributing to promote the school readiness of the child [9]. Therefore, studies on the school readiness of children with ASD will also investigate what resources the school, the family and the community need in order to provide a proper education suitable for children with ASD.

In our study, we developed a 14-item school readiness inventory for Korean children with ASD and demonstrated preliminary evidence of its content validity and reliability. The present study provides a basis for future studies that would further help evaluate and promote school readiness of the children with ASD.

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are not publicly available due to privacy and ethical restrictions, but are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Soyoung Irene Lee. Data curation: Hye In Jeong. Formal analysis: Jeewon Lee. Investigation: Jeewon Lee. Methodology: Soyoung Irene Lee. Software: HyunChul Yoon. Validation: Shin-Gyeom Kim. Writing—original draft: Hye In Jeong. Writing—review & editing: Jeewon Lee, Soyoung Irene Lee.

Funding Statement

This study was supported by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) (Grand Number 2022R1G1A1012424). This study was also supported by the Soonchunhyang University Research Fund.

References

1. American Psychiatric Association. Diagnostic and statistical manual of mental disorders 5thth ed. Arlington, VA: American Psychiatric Association; 2013.
2. Zwaigenbaum L, Bauman ML, Choueiri R, Kasari C, Carter A, Granpeesheh D, et al. Early intervention for children with autism spectrum disorder under 3 years of age: recommendations for practice and research. Pediatrics 2015;136(Suppl 1):S60–S81.
3. Marsh A, Spagnol V, Grove R, Eapen V. Transition to school for children with autism spectrum disorder: a systematic review. World J Psychiatry 2017;7:184–196.
4. Sparapani N, Morgan L, Reinhardt VP, Schatschneider C, Wetherby AM. Evaluation of classroom active engagement in elementary students with autism spectrum disorder. J Autism Dev Disord 2016;46:782–796.
5. Jahromi LB, Bryce CI, Swanson J. The importance of self-regulation for the school and peer engagement of children with high-functioning autism. Res Autism Spectr Disord 2013;7:235–246.
6. Eisenhower AS, Blacher J, Bush HH. Longitudinal associations between externalizing problems and student–teacher relationship quality for young children with ASD. Res Autism Spectr Disord 2015;9:163–173.
7. Park I, Gong J, Lyons GL, Hirota T, Takahashi M, Kim B, et al. Prevalence of and factors associated with school bullying in students with autism spectrum disorder: a cross-cultural meta-analysis. Yonsei Med J 2020;61:909–922.
8. Pellicano E, Kenny L, Brede J, Klaric E, Lichwa H, McMillin R. Executive function predicts school readiness in autistic and typical preschool children. Cogn Dev 2017;43:1–13.
9. Unicef. School readiness: a conceptual framework New York: United Nations Children’s Fund; 2012.
10. Copple C. Getting a good start in school Washington, DC: National Education Goals Panel; 1997.
11. Council on Early Childhood, ; Council on School Health. The pediatrician’s role in optimizing school readiness. Pediatrics 2016;138e20162293.
12. Perrin HT, Feldman HM, Huffman LC. Development and evaluation of a school readiness curriculum for pediatrics residents. MedEdPORTAL 2020;16:10976.
13. Ilan M, Meiri G, Manelis-Baram L, Faroy M, Michaelovski A, Flusser H, et al. Young autism spectrum disorder children in special and mainstream education settings have similar behavioral characteristics. Autism Res 2021;14:699–708.
14. Klubnik C, Murphy L, Campbell JM, Reed CB, Warner-Metzger CM. Assessing understanding of social awareness concepts in children with intellectual disability and autism spectrum disorder using the Bracken Basic Concept Scale—third edition. J Psychoeduc Assess 2014;32:157–164.
15. Janus M, Offord DR. Development and psychometric properties of the early development instrument (EDI): a measure of children’s school readiness. Can J Behav Sci 2007;39:1–22.
16. Pisani L, Borisova I, Dowd AJ. International development and early learning assessment technical working paper Washington, DC: Save the Children; 2015.
17. Gladstone M, Lancaster GA, Umar E, Nyirenda M, Kayira E, van den Broek NR, et al. The Malawi Developmental Assessment Tool (MDAT): the creation, validation, and reliability of a tool to assess child development in rural African settings. PLoS Med 2010;7e1000273.
18. Hughes C, Foley S, White N, Devine RT. School readiness in children with special educational needs and disabilities: psychometric findings from a new screening tool, the brief early skills, and support index. Br J Educ Psychol 2018;88:606–627.
19. Mukkiri S, Kandasamy P, Subramanian M, Chandrasekaran V, Kattimani S. Content validation of school readiness module and school readiness scale for assessing school readiness in children with autism spectrum disorder. Asian J Psychiatr 2022;71:103073.
20. Alonzi-Gold D, Grajo LC. The content validity and clinical utility of the kindergarten readiness inventory (K-READI): a screening tool of school readiness for children on the autism spectrum. J Occup Ther Sch Early Interv 2021;14:75–89.
21. Nair MKC, Radhakrishnan R, Olusanya BO. Promoting school readiness in children with developmental disabilities in LMICs. Front Public Health 2023;11:993642.
22. Kim SY, Oh M, Bong G, Song DY, Yoon NH, Kim JH, et al. Diagnostic validity of autism diagnostic observation schedule, second edition (KADOS-2) in the Korean population. Mol Autism 2022;13:30.
23. Oh M, Song DY, Bong G, Yoon NH, Kim SY, Kim JH, et al. Validating the autism diagnostic interview-revised in the Korean population. Psychiatry Investig 2021;18:196–204.
24. Ayre C, Scally AJ. Critical values for Lawshe’s content validity ratio: revisiting the original methods of calculation. Meas Eval Couns Dev 2014;47:79–86.
25. Streiner DL. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 2003;80:99–103.
26. Yen M, Lo LH. Examining test-retest reliability: an intra-class correlation approach. Nurs Res 2002;51:59–62.
27. MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods 1999;4:84–99.
28. Kim JW, Kim HW, Moon DS, Lim YS, McDougle CJ, Howe YJ. Comparison of services for autism spectrum disorder in Massachusetts with those in Seoul. J Korean Med Sci 2019;34e288.

Article information Continued

Table 1.

Demographic and clinical characteristics of the participants

Variable Value (N=22)
Sex
 Male 18 (81.81)
 Female 4 (18.18)
Mother educational level
 Graduated high school 3 (13.64)
 Graduated university or more 19 (86.36)
Father educational level
 Graduated high school 6 (27.27)
 Graduated university or more 16 (72.73)
Subjective economic status
 High 3 (13.64)
 Middle 13 (59.09)
 Low 6 (27.27)
Current educational setting
 Specialized class in kindergarten 12 (54.55)
 No specialized class in kindergarten 10 (45.45)
Plan for school
 Special education school 4 (18.18)
 Specialized class in mainstream school 11 (50.00)
 Regular class in mainstream school 7 (31.82)
K-WISC-V
FSIQ 59.41±23.92
CARS 35.20±3.32
ADOS-2
 Social affect total 14.90±4.30
 Restricted/repetitive behaviors 3.86±1.85
 Total 18.71±4.57
ADI-R
 Qualitative Impairments in reciprocal social interaction 21.86±3.91
 Qualitative abnormalities in communication 13.67±4.26
 Restricted, repetitive, and stereotyped patterns of behavior 5.86±1.91
 Abnormality of development evident at or before 36 months 3.62±1.07
K-SRI
 Self-help skills 16.45±3.23
 Social and emotional development 13.27±5.24
 School behavior 11.86±3.29
 Literacy and numeracy skills 6.91±2.71
 Total 46.77±9.76

Data are presented as mean±standard deviation or number (%). KWISC-V, Korean Wechsler Intelligence Scale for Children, Fifth Edition; FSIQ, Full Scale Intellectual Quotient; CARS, Childhood Autism Rating Scale; ADOS-2, Autism Diagnostic Schedule, Second Edition; ADI-R, Autism Diagnostic Interview, Revised; K-SRI, Korean School Readiness Inventory

Table 2.

Content validity of the K-SRI

Highly relevant Moderately relevant Minimally relevant Not relevant Mean CVR
Self-help skills
 Toilet training 9/9 0/9 0/9 0/9 4.00 1.00
 Self-feeding 9/9 0/9 0/9 0/9 4.00 1.00
 Removes/wears clothes/shoes 9/9 0/9 0/9 0/9 4.00 1.00
 Able to use objects 7/9 2/9 0/9 0/9 3.78 1.00
Social and emotional development
 Play participation 5/9 4/9 0/9 0/9 3.56 1.00
 Social conversation 7/9 2/9 0/9 0/9 3.78 1.00
 Turn taking 8/9 1/9 0/9 0/9 3.89 1.00
 Frustration tolerance 9/9 0/9 0/9 0/9 4.00 1.00
School behavior
 Sitting behavior 9/9 0/9 0/9 0/9 4.00 1.00
 Pays attention 6/9 3/9 0/9 0/9 3.67 1.00
 Follows directions 9/9 0/9 0/9 0/9 4.00 1.00
 Transitions 5/9 4/9 0/9 0/9 3.56 1.00
Literacy and numeracy skills
 Number counting 4/9 5/9 0/6 0/9 3.45 1.00
 Number writing 3/9 2/9 4/9 0/9 2.89 0.11
 Letter reading 2/9 6/9 1/9 0/9 3.12 0.78
 Letter writing 2/9 1/9 6/9 0/9 2.56 -0.33

K-SRI, Korean School Readiness Inventory; CVR, Content Validity Ratio

Table 3.

Reliability of the K-SRI

K-SRI subscales Cronbach’s alpha ICC (95% Cl)
Self-help skills 0.79 0.86 (0.73–0.94)***
Social and emotional development 0.79 0.79 (0.60–0.91)***
School behavior 0.85 0.68 (0.39–0.85)***
Literacy and numeracy skills 0.94 0.93 (0.83–0.97)***
***

p<0.001.

K-SRI, Korean School Readiness Inventory; ICC, intraclass coefficient; CI, confidence interval