Validity and reliability of the 10-Item Adverse Childhood Experiences Questionnaire (ACE-10) among adolescents in the child welfare system

Introduction Multiple evidence suggests that the vast majority of children in the Child Welfare System (CWS) are victims of early, chronic, and multiple adverse childhood experiences. However, the 10-item version of the Adverse Childhood Experiences Questionnaire (ACE-10) has never been tested in such a particularly vulnerable population as adolescents living in the CWS. We aimed to assess the psychometric properties of the ACE-10 in a community sample of 240 Hungarian adolescents placed in family style group care (FGC) setting. Methods Demographic data, the 10-item version of the Adverse Childhood Experiences Questionnaire (ACE-10), the Strengths and Difficulties Questionnaire (SDQ), and the HBSC Bullying Measure were used. Results Our results showed acceptable internal consistency (α = 0.701) and item-total correlations (rpb = 0.25–0.65, p < 0.001). However, our results also reflect that item 6 (“Parental separation/divorce”) is weakly correlated with both the cumulative ACE score and the rest of the questionnaire items. When item 6 is removed, the 9-item version of the ACE produces more favorable consistency results (α = 0.729). Strong and significant associations of the cumulative ACE score with emotional and behavioral symptoms and bully victimization confirm the concurrent criterion validity of both versions of the instrument. Discussion Our findings suggest that ACE-9 and ACE-10 are viable screening tools for adverse childhood experiences in the CWS contributing to the advancement of trauma-informed care. We recommend considering the use of either the 9-item or the 10- item version in the light of the characteristics of the surveyed population. The implications and limitations are discussed.


Introduction 1.Child maltreatment and its consequences
The social determinants of health are the conditions in which people are born, grow, work, live, and age, and the broader set of forces and systems shaping the conditions of daily life (1).The social determinants of health represent a broader concept, whereas child maltreatment covers its specific aspect related to the family context.Poverty, unemployment, low socio-economic status and resulting chronic stress, and family structure characteristics (e.g., divorce, household substance use, household mental illness) are risk factors for child maltreatment (2,3).The ecological model of the etiology of child abuse views maltreatment as a system of interacting risk and protective factors at four levels: the individual or ontogenic level, the family microsystem, the contextual level, and the social macrosystem (4).Accordingly, child maltreatment contributes to maintaining unfavorable social positions.
Child maltreatment (neglect, emotional, physical, sexual abuse) is a major public health issue worldwide.Over the past 20 years, studies were launched to explore child maltreatment on an increasingly broad scale.Some studies attempted to explore the prevalence of different forms of child abuse and neglect (5)(6)(7)(8)(9)(10), whereas others focused on exploring their risk factors (2,3,(11)(12)(13) and consequences (14)(15)(16).Recent research, however, has started examining a wider range of adverse factors affecting children; therefore, instead of using the term "child maltreatment, " we will use the broader term extended "adverse childhood experiences" (ACEs), which also includes dysfunctional family environments (household dysfunction) in addition to maltreatment (neglect and abuse) (15).Research has confirmed that adverse childhood experiences within the family are strong predictors of mental disorders and somatic outcomes, including chronic diseases (14)(15)(16).

Children in the child welfare system
Children in child protection are one of the most vulnerable populations, supporting their growth and providing them with psychological/social care is a major challenge worldwide (17).Children who are displaced from their biological families have a much more difficult life path (18).Children in CWS have a high prevalence of various adverse childhood experiences, ranging from 28 to 80% (19)(20)(21)(22)(23)(24)(25)(26).The different prevalence results of these studies are influenced by differences in definitions and groupings of adversities/trauma, differences in measurement instruments, as well as age differences in the sample (27).
The conceptual model ACE pyramid represents the life-long consequences of persistent adverse experiences in childhood (28).Early onset, cumulative and prolonged adverse experiences can result in disrupted neurodevelopment, immune and endocrine system modifications (29)(30)(31).Attachment difficulties, emotional and behavioral dysregulation result in a lack of appropriate problemsolving (or coping mechanism) strategies, leading some of those exposed to adversity to engage in persistent health-damaging behaviors (28,(32)(33)(34).Some of these children suffering from social, emotional and cognitive impairments will develop mental and somatic illnesses over their life course, which leads to disability and/or social impairments later in life (28, 31).

Measuring adverse childhood experiences
Understanding the prevalence and characteristics of adverse childhood experiences is essential for planning interventions aimed at reducing child maltreatment, this requires easy-to-use, reliable measurement tools (27,35,36).
ACEs can be measured through self-assessment questionnaires or expert interviews.The advantage of questionnaires is that they are economical, easy to administer and score, and ensure anonymity, which can reduce the chance of biased responses triggered by shame associated with trauma (35, 37).Self-report by adolescents is still found to be more reliable than agency records, parental reports of adolescent victimization or adult retrospective self-report (37,38).
Measures used to assess the prevalence of adverse childhood experiences should use consistent concepts about the types of child maltreatment and the same stands for their definitions (39).To avoid underestimation, items in measures should be behavior-specific, not ambiguous or non-specific (40).Ideally, all five forms of child maltreatment (physical abuse, sexual abuse, emotional abuse, neglect, and household dysfunction) should be measured simultaneously, as a number of children experience resulting polyvictimization and its increased consequences (16,41,42).
In a previous study methods to assess ACEs among children and families were identified and compared (35).Measuring tools commonly used in children and adolescents (without attempting to be comprehensive): Yale-Vermont Adversity in Childhood Scale; Adverse Childhood Experiences Questionnaire; Adverse Childhood Experiences Abuse Short Form (ACE-ASF); Philadelphia Childhood Adversity Questionnaire (CAQ); Childhood Trauma Questionnaire-Short Form (CTQ-SF); Juvenile Victimization Questionnaire (JVQ) (35,(43)(44)(45).
The 10-item ACE Score Calculator examines simultaneously all five forms of child maltreatment outlined above, with clear, unambiguous questions that are behavior-specific and concrete.In this way, the questionnaire is also suitable for testing an adolescent population in the child welfare system, so the clarity of the questions allows for the examination of children.To the best of our knowledge, only one study so far has examined the psychometric properties of this questionnaire in a normal adolescents population (46), although it is perfectly suitable for screening a larger population for adverse childhood experiences (as it can be applied simply and quickly).
The aim of our study was to assess the psychometric properties of the 10-item ACE Score Calculator and to demonstrate its reliability and validity in a specific sample of Hungarian adolescents placed in the Child Welfare System.There are no validated tools in Hungary for screening the vulnerable group of children living in CWS.The advances of the ACE-10 questionnaire mentioned above make it suitable for screening children in child welfare system for adverse childhood experiences.Increased attention to screening and assisting this population is needed, not only for the benefit of the population concerned, but also for the society as a whole.

Sampling and data collection
The data collection on adolescents (aged 12˗17) under child protection services was carried out in 31 family-style group care (FGC) settings in 3 counties of Hungary, and was conducted between March 2018 and January 2020.The sampling frame for the CWS sample consisted of 309 mentally sound adolescents living in 31 FGCs.After having been provided oral and written information, the adolescents and their guardians signed the informed consent form to participate in the study.Adolescents filled in the questionnaire anonymously.The questionnaires were filled in during group sessions, where each three adolescents were supervised by one health psychology Msc student, whose help was mainly needed in case of reading or attention difficulties.When emotional, cognitive, or other reasons made it necessary, the questionnaires were completed in individual sessions instead.A total of 271 adolescents completed the questionnaire.The reasons for the missing responses were that some adolescents were not at home on the day of data collection, were runaways, or either the guardians or the adolescents refused to provide consent.After data cleaning (elimination of incompletely filled questionnaires), the final sample size was 240.Ethics approval was issued by the Research Ethics Committee of the Hungarian Medical Research Council under the approval number ETT TUKEB 47848-7/2018/EKU.

Measures
The demographic data of the studied adolescents were assessed using a self-developed questionnaire with items asking for information on gender, age, nationality, school type, grade, and residential location.We administered the Strengths and Difficulties Questionnaire (SDQ) (47,48) and the HBSC (Health Behavior in School-aged Children Questionnaire) Bullying Measure (49,50) to assess concurrent criterion validity of the ACE Score Calculator.Multiple studies have demonstrated that the social, emotional, and behavioral symptoms and roles in bullying (perpetration/victimization) are significant correlates of the accumulation of childhood adversities (16,(51)(52)(53)(54)(55).
ACEs were assessed using 10-item version of the Adverse Childhood Experiences Questionnaire (ACE-10), which is a retrospective self-report questionnaire consisting of 10 items (56).The questions in this survey aim to assess 10 types of early ACEs suffered before the age of 18.These ACEs cover the possible forms of maltreatment (physical, emotional, and sexual abuse, and physical, emotional neglect) and household dysfunction (parental separation/ divorce, household physical violence, household substance abuse, household mental illness or suicide attempt, incarcerated household member).Previous research has demonstrated the different and distinct nature of these events through a series of analyses.This is why the questionnaire includes these items (15,56,57).
A cumulative ACE score between 0 and 10 is calculated by summing the number of 'yes' responses for each question, based on the number of types of ACEs.The ACE score is a severity index that measures the accumulation of different types of adverse experiences, showing how many types of adversities a person has experienced in their childhood.Our previous study confirms that the ACE-10 is suitable for assessing intrafamilial adverse childhood experiences in adolescents (46).The content of the items and the response options are provided in the Supplementary appendix.
For the assessment of social, emotional, and behavioral symptoms the Strengths and Difficulties Questionnaire (SDQ) was employed (47,48).The items of this questionnaire can be grouped in five factors as follows: hyperactivity, emotional problems, behavioral disorders, peer relationship problems, and prosocial conduct.The Hungarian version of the Strengths and Difficulties Questionnaire was adapted and validated by Birkás et al. (58).The questionnaire was found to have acceptable internal consistency in the sample (Cronbach's alpha = 0.70).
Bullying was assessed using the relevant questions of the Health Behavior in School-Aged Children Questionnaire (HBSC-2014) (49,50).The HBSC questionnaire is a comprehensive measure of health behaviors administered among school-aged children every 4 years.The HBSC study is based on a standardized methodology and conducted in more than 40 countries in international collaboration with the World Health Organization (59).The HBSC Bullying Measure includes questions on bullying, the role of perpetrators and victims of physical and emotional abuse and cyberbullying, and participants of physical fight.Previous studies have reported good reliability of scales (Cronbach's alpha = 0.76-0.84)(60,61).First, the phenomenon of bullying was introduced, followed by four questions about bullying (in-person bullying victimization, cyberbullying victimization, bullying perpetration, and physical fight).In all the four categories the answers to choose from were: never/once or twice/2 or 3 times a month/about once a week/ several times a week.After combining the categories of bullying variables, we defined frequency as a binary variable (never vs. at least once in the past 12 months) (61).

Statistical analysis
Statistical analyses were carried out using IBM SPSS Statistics ver.23.0 (IBM, Armonk, NY, United States).The sociodemographic and ACE characteristics, the mean and standard deviation of SEB symptom scores, and the frequency of bullying variables were described in the sample overall and by gender.Psychometric properties of the ACE Score Calculator were investigated through internal consistency (Cronbach's alpha calculation), intercorrelations (Phi correlation), item-total correlations (Point-biserial correlations), and association analyses for concurrent criterion validity.Since the studies conducted during the development and evolution of the test confirmed the distinct nature of the events tested in each item, the dimensionality of the test was not examined (15,56,57).
Generalized linear models with entry method were used to test associations of ACE score with SEB symptoms, and logistic regression models with backward (Wald) method for ACE score with bullying variables.All models were adjusted for age, gender, and location, and post-test analysis was carried out with the help of the adjusted Wald test.
The most frequent type of reported child maltreatment was emotional abuse in 32.1% (n = 76), and emotional neglect in 30.1% (n = 71) of the sample.The least prevalent reported child maltreatment was sexual abuse in 13.6% (n = 32) of the respondents.Parental divorce or separation was reported by 71.2% of the adolescents (n = 196), which was followed by incarcerated household member 48.5% (n = 114) and household substance abuse (32.5%, n = 76).These three dysfunctions were the most prevalent reported dysfunctional household conditions, while the least prevalent was having experienced household physical violence (21.5%, n = 51) (Table 1).
Table 2 presents the descriptive statistics of SEB symptoms and bullying variables in the sample.The mean scores of SEB symptoms ranged from 3.00 (SD = 1.60) (conduct problems) to 3.64 (SD = 1.80) (hyperactivity/inattention problems), while the mean for the total difficulties score was 12.81 (SD = 5.43).

Internal consistency and intercorrelations of the 10-item version of the ACE score calculator
The Cronbach's alpha reliability shows an acceptable internal consistency (α = 0.700).
Intercorrelations of ACEs (Table 3) were used to study the strength of associations between the frequency of occurrence of each adverse event.The item "Parental separation/divorce" barely correlated with the other types of ACEs, and the item "Sexual abuse" only correlated with four of the rest of the adverse events.

Item-total correlation
Point-biserial correlations were carried out to investigate the itemtotal correlation of the questionnaire.Table 4 shows that "Parental separation/divorce" was the only item to show weak correlation with the cumulative ACE score.In the case of the rest of the items, at least moderate correlations were found, which suggests the appropriate item-total correlations of the scale."Physical abuse" exhibited the strongest correlation with the cumulative ACE score.
Reviewing the data on intercorrelations and item-total correlations, the item "Parental separation/divorce" proved to be the least correlating item with the rest of the test items and the cumulative score.Consequently, we decided to examine the concurrent criterion validity of the test when the ACE 6 item ("Parental separation/ divorce") is removed.
When calculating the Cronbach's alpha reliability index of the 9-item version, we found a more favorable value than in the case of the 10-item version (α = 0.729).

Discussion
In the present study, we conducted the psychometric evaluation of the ACE-10 questionnaire in a sample of Hungarian adolescents in the CWS.We assessed the intercorrelations, item-total correlations, concurrent criterion validity, and reliability of the instrument.The 10-item short form of the original ACE questionnaire comprised questions measuring 5 types of maltreatment (physical, emotional, sexual abuse, physical, and emotional neglect), and 5 types of household dysfunction.

Reliability and validity
Consistently with our study on average adolescent population (46), the ACE-10 showed appropriate internal consistency and reliability with a Cronbach's alpha of 0.70.For internal correlation, we found that sexual abuse is only correlated with dysfunctional family factors (apart from physical neglect).This may be explained by the fact that sexual abuse is the only item that refers not only to harm suffered within the family but also to abuse suffered outside the family (15).The CWS population is an at-risk population, where a severely dysfunctional family system increases the risk of exposure to harm outside the family.Accordingly, we can assume that the "yes" responses for these item do not only refer to events experienced within the family, as opposed to the rest of the items.The item "Parental separation/divorce" is only correlated with the item "incarcerated household member, " and the correlation there is only a weak one.The results found for these two items are not in line with the results of our study on the average population, where both "Sexual abuse" and "Parental separation/divorce" were correlated with the other items of the scale.
The ACE-10 scale assessed in the CWS population showed adequate item-total correlations.However, not all of the ACE-10 items were found to be equally relevant in the Point-biserial analysis.Again, it is the item "Patental separation/divorce" that barely correlated with the ACE cumulative score.This result differs from our study on the average sample, where this item showed a moderate correlation with the ACE score.
A possible explanation for the less favorable psychometric indicators of this item may be found in the wording of the item itself: "Were your parents ever separated or divorced?"This is meant to provide information on the fact of separation itself, but eventually frontiersin.orgraises other relevant questions.Did the parents ever live together?How old was the child when the parents separated?How old was the child when he/she was placed in CWS?When did the parents separate compared to the time of the child's placement in CWS? Was this a high-conflict separation/divorce or not (62)?Was it the parents' cohabitation or their separation that had a major contribution more on the child's placement in CWS?It is worth noting that 71% of the adolescents in the sample reported having separated parents.This means that the sample can be considered as almost homogeneous in this regard, which may also have contributed to the results described above.
In addition, it is important to note that Hungary has relatively high divorce rates (1.9-2.2/1,000people) in the European Union (63).It is also known that divorce rates are higher among people of lower socio-economic status (52), which is the social segment from which the majority of children in CWS across Europe are reported (53).This raises the question whether parental separation/divorce as an adverse experience should be asked about in this population at all, and if the psychometric properties of the test were still adequate if the item was omitted.
Accordingly, in the next stage we removed the sixth item and checked the psychometric properties of the 9-item version of the ACE.Our results show that the 9-item version of the test yielded better psychometric properties on the CWS population.In the light of the results, we recommend the careful use of item 6 ("Parental separation/divorce").We suggest that the relevance of this item should be considered after careful examination of the population to be studied.However, our results also indicate that both the 10-item version or the ACE 9-item version of ACE can provide reliable information with appropriate psychometric properties.
The concurrent criterion validity of the instrument was good in our sample.We found strong and cumulative associations of the total ACE score with emotional and behavioral symptoms and bully victimization.The prevalence and gender distribution of reporting bully perpetration and victimization were similar to those found in other studies, in which girls also dominated in both cases (64).However, contrary to our expectations, the severity of peer relationship problems was independent from the cumulative ACE score.Given the previous evidence on clear correlation between ACE accumulation and the severity of peer problems in the general adolescent population (55), the setting of the CWS may be a clear explanation for our findings.Price and Brew (65) conclude that displacement and transitions themselves present unique social challenges for children.These transitions force them to adapt to new social expectations, such as fitting in with a new peer group.When placed in structured settings like group care, their interactions with peers are limited.These can exacerbate social difficulties that are already present as a result of maltreatment.Taking all these into consideration, the problems of peer relationships are very much related to transitions and dependent on the social context, equally for those reporting fewer or more ACEs.ACEs are individual experiences, but the transition adolescents face and the challenges it brings are collectively present, and collectively make it difficult for these youngsters to build and maintain proper relationships.

Strengths and limitations
Our study respondents were adolescents, which was advantageous for our study: they are closer in time to the period when they experienced the adversities compared to adults.Although children and adolescents may generally differ from adults in their ability to understand long-term consequences and regulate behavior, their cognitive abilities are not significantly different from those of adults (66)(67)(68).Similarly, adolescents' cognition and reliable episodic memory are sufficiently developed to allow their participation in this type of research (69, 70).The advantages of using scores to measure adverse childhood experiences lie in the simplicity of the scores, which facilitates their widespread application in policy, public health and clinical settings.It is particularly important to use an internationally accepted measure to focus attention on this population.
On the other hand, the disadvantage of using a cumulative score is that it fails to take into account the fact that there are other indicators of severity in addition to ACE cumulation, which may be particularly important when assessing children in child protection.The non-representativity of the sample allows no extrapolation of the results, and the sensitive nature of the topic may cause bias and can potentially reduce the willingness of reporting ACEs.Nevertheless, self-report by adolescents is still found to be more reliable than agency records, parental reports of adolescent victimization or adult retrospective self-report (37,38).A psychometric limitation of our study is that in the absence of other validated trauma questionnaires in Hungarian, convergent validity could not be assessed.In addition, the reliability levels, although acceptable, are at the limit of acceptability.The number of items and sample size also suggest caution.
A further limitation to the results may be that adolescents living in CWS may not recognize the adversities as adversities, as they have been socialized in them.Furthermore, in their case, self-reporting may have a biasing effect as they may wish to be reunited with their family of origin.Also, dissociation and memory deficits may bias the results downwards (71).

Conclusion
The results indicate that both the 10-item version and the 9-item version (without item 6) of the ACE is a valid and reliable measure of

TABLE 1
Demographic and ACE characteristics of the sample, overall and by gender.
a The category does not add up to the full sample size due to some missing data.

TABLE 2
Social, emotional, and behavioral (SEB) symptoms and bullying characteristics of the sample overall and by gender.

TABLE 4
Item-total correlations of the ACE questionnaire on the studied adolescent sample.Point-biserial correlation coefficients; ***p < 0.001.adversities among disadvantaged adolescents living in CWS.We suggest that the relevance of item 6 ("Parental separation/ divorce") should be considered in relation with the population under study.Screening for trauma among children in CWS and detailed investigation of trauma types is of utmost importance in CWS, as targeted interventions can be based on these data.In order to follow the principles of trauma-informed care (72, 73), a valid measurement tool is needed as a first step.The ACE questionnaire is a brief, timeand cost-efficient, easy-to-understand instrument, which makes it suitable for disadvantaged children, considering that early impairment can lead to reading, comprehension and learning difficulties (24, 31, 74).Its suitability for repeated testing is another advantage.We see it as an appropriate tool for the purposes of screening, research, and treatment planning.
a childhood