Reliability and validity of the repetitive behavior scale-revised for young Chinese children with autism spectrum disorder in Jiangxi Province

Introduction

Autism spectrum disorder (ASD), defined as a neurodevelopmental disease, has two core symptoms including social communication deficits and restricted and repetitive behaviors (RRBs) and sensor stimuli behaviors (1). Genes, environment, and the interaction between genes and environment are the etiology causes of ASD (2). The Autism and Developmental Disabilities Monitoring (ADDM) network estimated the prevalence of ASD among children aged 8 years, showing 0.66% in 2002 (3), 1.46% (4) in 2012, 1.68% (5) in 2014, and 1.85% (6) in 2016, respectively. A Chinese nationwide multi-center population-based study (7) recruited 125,806 participants aged 6–12 years to determine the prevalence of ASD, revealing that 0.7% of these children were diagnosed with ASD. This figure was higher than the 0.118% reported in a meta-analysis conducted in China (8), which included 25 studies correlated with the prevalence of ASD. As the prevalence of autism increases, there is an increasing focus on the condition around the world.

Restricted and repetitive behavior is a series of behaviors characterized by high frequency and invariant repetition; this results in the child desiring environmental sameness (9). RRBs have four subtypes of behavior in the Diagnostic and Statistical Manual of Mental Disorders 5th edition (DSM-5). The first is the repetition of actions, speech, or use of objects, such as turning around and parroting. The second is insisting on sameness such as walking the same route. The third behavior is restricted interests, including obsessing with wheels. The last one is sensory processing abnormalities, for example, smelling people’s hair (10). It is well known that the social communication of ASD has gained considerable research attention over the years; however, the field of RRBs lags far behind the former. In many infants later diagnosed with ASD, RRBs are detectable after 12 months of age, which may be some of the earliest detectable behavioral markers of ASD (11). RRBs are also present in typically developing (TD) infants, toddlers, and developmental delay (DD) children (12). Through literature review, we found that most of the measurements of RRBs were derived from some items of the Autism Diagnostic Interview-Revised (ADI-R) (13) and the Autism Diagnostic Observation Schedule (ADOS) (14) and a scale named the Repetitive Behavior Scale (Revised Edition), a 43-item informant-based rating scale, was used to assess RRBs only. This scale includes six subscales: (a) Rituals, (b) Self-injurious Behavior, (c) Stereotypic Behavior, (d) Compulsive Behavior, (e) Restricted Interests, and (d) Sameness. The RBS-R aimed to measure the presence and severity of a variety of RRBs and its items were drawn from other instruments and conceptually categorized based on the authors’ clinical experience (15). Numerous investigations have been conducted on the psychometric properties of the RBS-R including in Canada (16), German (17), Italy (18), Spain (19), United States (20), and so on. RBS-R was adopted in a study (21) of 163 Chinese children aged 3–8 with ASD enrolled, indicating that a 5-factor model was more appropriate for evaluating RRBs, with a Comparative Fit Index (CFI) of 0.99 and a Goodness of Fit Index (GFI) of 0.73. Another reliability and validity study (16) of RBS-R from Canada showed that the goodness-of-fit indices of the hypothesized latent-factor models of the RBS-R failed to achieve anticipated success. Another study in the United States found that the four- and six-factor models both demonstrated adequate-to-good fit in a sample of 350 children with ASD aged 2 to 9 years (20). In conclusion, the psychometric analysis of the RBS-R was confirmed with different nations and samples.

Since the RBS-R is a widely used analytical tool for researching the RRBs of ASD, it is crucial to clarify the appropriateness of using the RBS-R in Chinese children with ASD. Currently, there are only two domestic studies on the reliability and validity of RBS-R in ASD. In our study, we aimed to increase the sample to measure the reliability and validity of the RBS-R among children with ASD aged 2–4 years in Jiangxi province of China. According to Rouquette A (22), a minimum sample size of 300 subjects is generally acceptable but should be increased when the number of factors is large. This research was also designed to determine the feasibility of employing the RBS-R as an assessment tool for the identification and diagnosis of ASD in the future.

Materials and methods

Participants and procedures

We recruited children aged 2–4 years old from a hospital in Jiangxi province. Inclusion criteria were: (1) native Jiangxi province children, (2) no parent-reported visual and auditory impairment, epilepsy, cerebral palsy, trisomy 21 syndrome, or other psychiatric diseases. These children were selected from those who received the assessment of the Childhood Autism Rating Scale (CARS) and based on a score of no less than 30. Children were then diagnosed with ASD according to the DSM-5. The parents or caregivers of eligible children were given written informed consent and the demographic questionnaire, RBS-R, and Autism Behavior Checklist (ABC). The researchers verified their eligibility once they completed these questionnaires, and a second RBS-R was issued to the parents who consented to complete the questionnaire a second time one week after the first one was completed. When conducting the assessment, researchers would prompt participants to complete any incomplete questionnaires.

Measures instrument

The original Repetitive Behavior Scale (RBS) had three subscales: (a) Stereotypic Behavior, (b) Self-injurious Behavior, and (c) Compulsions which was established by Bodfish (23). RBS-R-based RBS was built by Bodfish in 2020 (15), which was intended to assess the variety of RRBs observed in individuals with ASD and the 43 items can be categorized into the six following subscales: stereotyped behavior (6 items), self-injurious behavior (8 items), compulsive behavior (8 items), ritualistic behavior (6 items), sameness behavior (11 items), and restricted behavior (4 items). All items are rated on a four-point Likert scale of severity; the higher the scores, the more severe the conditions (i.e., 0 = behavior does not occur, 1 = behavior occurs and is a mild problem, 2 = behavior occurs and is a moderate problem, 3 = behavior occurs and is a severe problem). The Chinese version was cited by Li et al. (24) and translated from English to Simplified Chinese by two native Chinese speakers who majored in developmental-behavioral pediatrics and were also fluent in English. To guarantee the consistency of the message, two professors then proofread and translated the Chinese version back into English. In our study, we added descriptions of some items by utilizing more pertinent and detailed explanations from knowledgeable doctors who evaluated numerous individuals with ASD. The RBS-R used in this study was completed by the parents/caregivers of the participants during the evaluation of CARS. The questions on this scale were given to the parents/caregivers to respond to with reference to the preceding month.

Data analysis

Confirmatory factor analysis (CFA) was applied to test the overall fit of the data to the scale model with 43 items. Statistical indices used to evaluate the model fits include the model of chi-square/degrees of freedom (χ2/df), the comparative fit index (CFI), the goodness of fit index (GFI), the root mean square error of approximation (RMSEA), the standardized root means square residual (SRMR), and the Akaike information criterion (AIC). The index criteria for well-fitting models were: CFI > 0.9, GFI > 0.9, RMSEA < 0.08, and 2 < χ2/df < 5 (25). The reliability of the RBS-R was determined by the test-retest reliability and internal consistency. Test-retest reliability was estimated by an intra-class coefficient (ICC), which was calculated by the correlation between the first and second completion of the scale. The recommended ICC value is ≥0.7, and a value of ≥0.6 is considered acceptable (26). Cronbach’s alpha was used to determine the internal consistency of the scale. An acceptable cutoff value is not below 0.7 (27). The CFA was performed using AMOS, version 17, and SPSS, version 19 was used for other statistical analyses.

Result

Demographic characteristics

The sample of the current study consisted of 381 Chinese children with ASD, including 305 boys and 76 girls, with a mean age of 3.14 years old (range 2–4.92). The mean CARS score was 36.04 (range 30–44). The descriptive statistics for these subjects, including both demographic and diagnostic information, are shown in Table 1.

TABLE 1

Table 1. Socio-demographic and diagnostic characteristics.

Item analysis

Table 2 provides details on the positive response indices, frequency of endorsement, and spearman’s correlation coefficients between the components of the total score. The frequency of positive responses ranged between 1.31% (i.e., for item 13: Inserts finger/object) and 77.17% (i.e., for item 4: locomotion). The correlation between items of total score ranged between 0.123 and 0.487.

TABLE 2

Table 2. Frequency of endorsement (affirmative answer) for items of the RBS-R and the spearman’s correlation coefficients between items of the total score.

Confirmatory factor analysis

We hypothesized a range of three to six factors for the specific models, as shown in Figure 1.

FIGURE 1

Figure 1. The 3–6 factors competing hypothesized models of the RBS-R factor structure.

The CFA results for all four models (i.e., Models 3, 4, 5, and 6) in Table 3, indicated that the 3-, 4-, 5-, and 6-factor models were all reasonably good fits for the data, based on the fit statistics guidelines previously described. However, considering the optimal statistical fit, we suggested the 6-factor to be the best model. The standardized factor loadings of CFA, ranging from 0.094 to 0.658, are shown in Table 4.

TABLE 3

Table 3. Goodness-of-fit indices for the hypothesized latent factor RBS-R models (N = 381).

TABLE 4

Table 4. The standardized factor loadings of the confirmatory factor for RBS-R.

Reliability statistics

Table 5 depicts the internal consistency (Cronbach’s a) indices and test-retest correlations for the RBS-R and each factor. The internal consistency for the total score on the RBS-R was 0.834, and the internal consistency for the factors ranged from 0.744 to 0.83. The aforementioned findings showed that this scale has strong internal consistency. A retest was conducted on 24 participants who had previously taken the RBS-R in order to evaluate the instrument’s consistency and reliability The test-retest reliability was good to excellent for the total score of the RBS-R (0.749) and each factor (factor 1: 737, factor 2:0.829, factor 3:0.639, factor 4:0.754, factor 5:0.607, and factor 6:0.689). As for the reliability of the RBS-R, the results indicated a high internal consistency of all subscales and the total score of the RBS-R in this study.

TABLE 5

Table 5. The internal consistency and test-retest reliability of the RBS-R.

Discussion

This study aimed to evaluate the psychometric properties of the RBS-R in patients with ASD in Jiangxi Province. If the RBS-R has strong validity and reliability, it will be used to gauge the severity of RRBs in patients with ASD and our findings confirmed the utility of the RBS-R as a measure of a wide range of RRBs seen in 2–4 years old children with ASD from Jiangxi province of China. This study supports Li’s (21) assertion that the original RBS-R items may be modified to fit the Chinese context.

The Self-Injurious behaviors (33.86%) exhibited the lowest frequency behaviors compared to other subscales in our study, which was consistent with most studies from other countries such as China (21) and America (20). However, the frequencies were much lower for each item of this subscale compared to the results of He et al. (21) with 12 (rubs/scratches) of 1.84% in our study and 13.5% in other Chinese participants (21), 18.5% in Spanish participants (19), 16.3% in United States participants with ASD (20), and 2.7% in Italian participants with ASD (18). These behaviors of item 9 (hits w/object), item 11 (Pulls hair/skin), item 12 (rubs/scratches), and item 13 (inserts finger/object) were infrequent during the assessment of CARS according to the reports from parents or caregivers. The highest frequency of item was item 4 (locomotion), which was 77.17% in the study. Among Spanish-speaking patients with ASD, the highest frequency was 50.2% from item 28 (communication) (19). The frequency of item 28 (communication) in our report was 65.62%, higher than the former. In an American study, the highest frequency was 67.7%, from item 40 (preoccupation with one subject) (20), suggesting that different samples and cultures can explain the differences in frequencies.

We proposed the structural models with three to six factors according to some literature (16). In accordance with the statistical indices’ guidelines for the structural equation model, these models were all reasonably good fits for our sample. The best model that fit well was the 6-factors model with the ratio between χ2 and the degrees of freedom (χ2/df) of 1.68, the CFI of 0.754, the GFI of 0.852, and the RMSEA of 0.042. The standardized factor loading ranged from 0.094 to 0.658, some items of which were lower than the cutoff. According to Kline’s criterion, a factor loading cutoff of ≥0.35 was applied (28). However, the index did not become much better after removing these three items, therefore, we reserved item 28 (communication), item 12 (rubs/scratches), and item 21 (repeating), which would be found in ASD children. Additionally, several of the items on this scale overlapped with one another. For example, parents were puzzled by item 31 (no interruption) and item 37 (difficult transitions) as item 31 came along with item 37. Coincidentally, scholars (21) found there were duplicates among the content of certain RBS-R items among the ritualistic subscale and sameness subscale.

The structure validity and internal validity of this scale were accessed using a confirmatory factor in accordance with the theoretical framework proposed by Bodfish (15) and other works of research. Concerning the reliability, the internal consistency for the subscales of the original structure ranged from 0.744 (Compulsive Behavior) to 0.83 (Stereotypic Behavior); the internal consistency for all RBS-R items was 0.834. Test-retest of the RBS-R ranged from 0.607 (Self-injurious subscale) to 0.829 (Self-injurious subscale); the test-retest for all RBS-R items was 0.749. These results indicated an acceptable internal consistency of all subscales and the total score of the RBS-R in this study. The internal consistency estimated by Cronbach’s alpha value was lower than a study by He et al. (21) and higher than another study by Li et al. (24). The diverse samples used in the experiments may be the cause of the inconsistent outcomes. But according to these three investigations, the RBS-overall R’s score and all of its subscales demonstrated acceptable levels of internal consistency.

Strengths and limitations

The first of the two strengths of our study is that we recruited 381 ASD children from 2 to 4 years old to ensure the sample size was sufficient to support the use of CFA, which was more than fivefold the number of items. Second, we collected these data of RBS-R while the assessment of CARS, so that evaluators were able to assess RRBs of ASD children face-to-face and we would check the scale to ensure that caregivers had finished the questionnaire, leading to reduced reporting errors from caregivers. The first limitation of our study was that we only obtained children 2–4 years old, in which, the behaviors from the compulsive subscale and sameness subscale may not function effectively. Secondly, we had not determined how this scale’s psychometric characteristics varied according to factors like age, gender, cognitive capacity, and so on, since is known to be age-related and dependent on the intellectual functioning of individuals with ASD (29).

Conclusion

In conclusion, using the original framework of RBS-R to assess the severity of repetitive and stereotypic behaviors in young Chinese children with ASD from Jiangxi Province would enable a better differential diagnosis. Future studies should expand the sample, focusing in particular on Chinese school-aged children with ASD.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by the Ethics Committee of Jiangxi Provincial Children’s Hospital. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

XL, XZ, and SZ designed the study. XL, YX, MG, LH, and ZL performed the study. XL wrote and reviewed the manuscript. All authors contributed to the article and approved the submitted version.

Acknowledgments

We thank all the children, parents, and caregivers who participated in this project.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2022.939841/full#supplementary-material

References

1. Lord C, Elsabbagh M, Baird G, Veenstra-Vanderweele J. Autism spectrum disorder. Lancet. (2018) 392:508–20. doi: 10.1016/S0140-6736(18)31129-2

CrossRef Full Text | Google Scholar

2. Masi A, DeMayo MM, Glozier N, Guastella AJ. An overview of autism spectrum disorder, heterogeneity and treatment options. Neurosci Bull. (2017) 33:183–93. doi: 10.1007/s12264-017-0100-y

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Van Naarden Braun K, Pettygrove S, Daniels J, Miller L, Nicholas J, Baio J, et al. Prevalence of autism spectrum disorders–autism and developmental disabilities monitoring network, 14 sites, United States, 2002. MMWR Surveill Summ. (2007) 56:12–28.

Google Scholar

4. Christensen DL, Baio J, Van Naarden Braun K, Bilder D, Charles J, Constantino JN, et al. Prevalence and characteristics of autism spectrum disorder among children aged 8 years–autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveill. Summ. (2016) 65:1–23. doi: 10.15585/mmwr.ss6503a1

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Baio J, Wiggins L, Christensen DL, Maenner MJ, Daniels J, Warren Z, et al. Prevalence of autism spectrum disorder among children aged 8 years – autism and developmental disabilities monitoring network, 11 sites, United States, 2014. MMWR. Surveill Summ. (2018) 67:1–23. doi: 10.15585/mmwr.ss6706a1

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Maenner MJ, Shaw KA, Baio J, Washington A, Patrick M, DiRienzo M, et al. Prevalence of autism spectrum disorder among children aged 8 years – autism and developmental disabilities monitoring network, 11 sites, United States, 2016. MMWR Surveill Summ. (2020) 69:1–12. doi: 10.15585/mmwr.ss6903a1

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Zhou H, Xu X, Yan W, Zou X, Wu L, Luo X, et al. Prevalence of autism spectrum disorder in China: A nationwide multi-center population-based study among children aged 6 to 12 years. Neurosci Bull. (2020) 36:961–71. doi: 10.1007/s12264-020-00530-6

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Sun X, Allison C, Matthews FE, Sharp SJ, Auyeung B, Baron-Cohen S, et al. Prevalence of autism in mainland China, Hong Kong and Taiwan: A systematic review and meta-analysis. Mol Autism. (2013) 4:7. doi: 10.1186/2040-2392-4-7

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kanner LJNC. Autistic disturbances of affective contact. Nervous Child. (1943) 35:100–36.

Google Scholar

10. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (DSM-5). 5th ed. Washington DC: American Psychiatric Association (2013). doi: 10.1176/appi.books.9780890425596

CrossRef Full Text | Google Scholar

11. McKinnon CJ, Eggebrecht AT, Todorovet A, Wolff JJ, Elison JT, Adamset CM, et al. Restricted and repetitive behavior and brain functional connectivity in infants at risk for developing autism spectrum disorder. Biol Psychiatry Cogn Neurosci Neuroimaging. (2019) 4:50–61. doi: 10.1016/j.bpsc.2018.09.008

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Sifre R, Berry D, Wolff JJ, Elison JT. Longitudinal change in restricted and repetitive behaviors from 8-36 months. J Neurodev Disord. (2021) 13:7. doi: 10.1186/s11689-020-09335-0

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Cantor RM, Navarro L, Won H, Walker RL, Lowe JK, Geschwind DH. ASD restricted and repetitive behaviors associated at 17q21.33: Genes prioritized by expression in fetal brains. Mol Psychiatry. (2018) 23:993–1000. doi: 10.1038/mp.2017.114

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Troyb E, Knoch K, Herlihy L, Stevens MC, Chen CM, Barton M, et al. Restricted and repetitive behaviors as predictors of outcome in autism spectrum disorders. J Autism Dev Disord. (2016) 46:1282–96. doi: 10.1007/s10803-015-2668-2

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Bodfish JW, Symons FJ, Parker DE, Lewis MH. Varieties of repetitive behavior in autism: Comparisons to mental retardation. J Autism Dev Disord. (2000) 30:237–43. doi: 10.1023/A:1005596502855

CrossRef Full Text | Google Scholar

16. Mirenda P, Smith IM, Vaillancourt T, Georgiades S, Duku E, Szatmari P, et al. Validating the repetitive behavior scale-revised in young children with autism spectrum disorder. J Autism Dev Disord. (2010) 40:1521–30. doi: 10.1007/s10803-010-1012-0

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Kästel IS, Vllasaliu L, Wellnitz S, Cholemkery H, Freitag CM, Bast N. Repetitive behavior in children and adolescents: Psychometric properties of the German version of the repetitive behavior scale-revised. J Autism Dev Disord. (2021) 51:1224–37. doi: 10.1007/s10803-020-04588-z

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Fulceri F, Narzisi A, Apicella F, Balboni G, Baldini S, Brocchini J, et al. Application of the repetitive behavior scale-revised–Italian version–in preschoolers with autism spectrum disorder. J Autism Dev Disord. (2016) 48:43–52. doi: 10.1016/j.ridd.2015.10.015

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Martínez-González AE, Piqueras JA. Validation of the repetitive behavior scale-revised in spanish-speakers participants with autism spectrum disorder. J Autism Dev Disord. (2018) 48:198–208. doi: 10.1007/s10803-017-3276-0

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Hooker JL, Dow D, Morgan L, Schatschneider C, Wetherby AM. Psychometric analysis of the repetitive behavior scale-revised using confirmatory factor analysis in children with autism. Autism Res. (2019) 12:1399–410. doi: 10.1002/aur.2159

PubMed Abstract | CrossRef Full Text | Google Scholar

21. He H, Ye N, Yi L, Yang C. Validating the repetitive behavior scale-revised for children in China aged 3 to 8 with autism spectrum disorder. J Autism Dev Disord. (2019) 49:4941–56. doi: 10.1007/s10803-019-04210-x

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Rouquette A, Falissard B. Sample size requirements for the internal validation of psychiatric scales. Int J Methods Psychiatr Res. (2011) 20:235–49. doi: 10.1002/mpr.352

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Bodfish JW, Symons FW, Lewis MH. The repetitive behavior scale. Chapel Hill, NC: Western Carolina Center Research Reports (1998).

Google Scholar

24. Li J, Jiang Z, Cui G, Guo L, Zhun J. Reliability and validity of the Chinese version of the repetitive behavior ScaleRevised (RBS-R) in children with autism. Chin Pediatr Integr Tradit Western Med. (2013) 5:208–11.

Google Scholar

25. Hou F, Qi L, Liu L, Luo X, Gu H, Xie X, et al. Validity and reliability of the Dyslexia checklist for chinese children. Front Psychol. (2018) 9:1915.

Google Scholar

26. Aleksic J, Zvekic-Svorcan J, Stupar NV, Jeremic I, Grgurevic AJM. Cross-cultural validation of the modified falls efficacy scale in Serbian community-dwelling women at risk for osteoporotic fracture. Menopause. (2018) 25:444–50. doi: 10.1097/GME.0000000000001009

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. (2011) 2:53–5. doi: 10.5116/ijme.4dfb.8dfd

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Quah PL, Cheung YB, Pang WW, Toh JY, Saw SM, Godfrey KM, et al. Validation of the children’s eating behavior questionnaire in 3 year old children of a multi-ethnic Asian population: The GUSTO cohort study. Appetite. (2017) 113:100–5. doi: 10.1016/j.appet.2017.02.024

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Esbensen AJ, Seltzer MM, Lam KS, Bodfish JW. Age-related differences in restricted repetitive behaviors in autism spectrum disorders. J Autism Dev Disord. (2009) 39:57–66. doi: 10.1007/s10803-008-0599-x

PubMed Abstract | CrossRef Full Text | Google Scholar