Original Research ARTICLE
Evaluation of the factor structure of the Obstacles to Engagement Scale with low-income African American parents
- 1Department of Pediatrics, University of Louisville School of Medicine, Louisville, KY, USA
- 2Department of Pediatric Psychology and Neuropsychology, Nationwide Children’s Hospital, Columbus, OH, USA
- 3Department of Pediatrics, The Ohio State University College of Medicine, Columbus, OH, USA
- 4University of Louisville School of Nursing, Louisville, KY, USA
- 5University of Louisville Hospital, Louisville, KY, USA
Objective: Parenting anticipatory guidance is one way to promote optimal child health and development and minimize disparities between children from lower socio-economic status families and their higher income peers. However, low rates of attendance at and completion of parenting programs has been demonstrated. Understanding barriers to participation has important implications. The Obstacles to Engagement Scale (OES) has been used in some populations but it has not been evaluated for use with low-income African American samples. The aim of the current study is to evaluate the factor structure of the OES with a sample of low-income, African American parents.
Method: Parents or legal guardians with children aged 3–8 years completed a survey in the waiting room of a primary care pediatric academic practice in an urban location in the southern United States of America (N = 114). Almost 87% had <12th grade education and 93% of the children received Medicaid services. The OES was one measure from a larger study and only participants with complete data on the OES were included in the exploratory factor analysis (EFA).
Results: The EFA did not support the previous 4-factor solution (intervention demands, personal or family stressors or obstacles, relevance of or trust in intervention, and time and scheduling demands. Instead, a 3-factor statistical solution emerged but not all items held together conceptually.
Conclusion: The current study supports the necessity for evaluating study instruments for use with specific populations. Larger samples are needed to disentangle the effects of educational and poverty status from race and ethnicity and to develop and validate instruments that are appropriate for the study population.
Early childhood is a critical period for laying the foundation for later child health, behavior, and development (1–8). The quality of parent–child interactions during early childhood has important implications for long-term health, development, and behavioral outcomes and for health and educational resource utilization (1, 9–11). Risk factors for less than optimal child development include poverty, minority status, and low levels of parent education (1, 12, 13). There are many reasons why low-income, ethnic minority families may need additional support in establishing positive parenting skills, including positive parent-child interactions, such as lack of knowledge about health and development, lower health literacy, higher levels of stress, and fewer available resources (3, 14–18). More data are needed to understand what specific factors serve as barriers for parents in receiving the support they need.
Parenting programs have been shown to be successful in improving child outcomes. With early intervention and prevention, many negative consequences can be avoided and further psychosocial dysfunction can be prevented (19). However, much remains unknown about the factors that serve as barriers for parents’ help-seeking to maximize child health and development and to prevent adverse outcomes. Parental attendance, in general, has been low for parenting programs even when their children have documented or perceived behavior problems (20–22). The Obstacle to Engagement Scale (OES) has been used in studies to understand help-seeking behavior of parents for their children’s behavior (20, 23). Data obtained while using the tool has potential to inform the development of interventions aimed at engaging parents and removing real or perceived barriers for their participation in programs that promote the development of positive parenting skills. However, care must be taken in using such instruments that have been developed and validated for other populations without validating them for other populations of interest.
Understanding barriers to parental help-seeking may lead to the development of targeted messages to increase service utilization and reduce health disparities in children who are at greatest risk such as those from low-income families and ethnic minorities. Barriers may differ with parents who differ in socioeconomic and/or educational status, race/ethnicity, age, gender, and health status, to name a few. OES (20) is a measure of potential barriers to treatment that was specifically developed to assess barriers to participation in preventative behavioral parenting classes for parents of preschool-age children. The original sample was described as economically and ethnically diverse (20) with just over half self-identifying as African American. The sample was recruited from day care centers in two metropolitan cities in the United States (N = 410). Others have validated the scale in samples that were predominately Caucasian (24) who were recruited from pediatric primary care settings in the Midwestern United States. To date, no studies have been found that specifically examine the psychometric properties of the scale when used with low-income African American parents. Therefore, the current study aims to determine if the factor structures previously described in other groups are the same as that found when the OES is used with a sample of low-income African American parents. Administration of the questionnaire asked parents about reasons that might prohibit their attendance at preventative parenting classes.
Materials and Methods
As part of a larger study, a sample of 190 parents or legal guardians completed a survey in the waiting room of a primary care pediatric academic practice that serves predominantly low-income families in an urban location in the southern United States of America. Data collection took place May through August 2011. All primary caregivers with children aged 3–8 years were invited to participate. For the larger study, parents were asked about their preferences for receiving parenting information and about issues related to stigma in addition to their obstacles to engagement. For the current study we focused on the African American participants who had complete data from the OES (N = 114). See Table 1 for demographic information on participants.
Parents completed a survey containing demographic characteristics of themselves and their children. Additionally, parents also completed the OES (barriers to treatment) (20). The instructions for the OES portion of the survey were as follows. “There are many reasons why parents might choose to attend or not to attend parenting classes. Listed below are some reasons that might keep parents from attending. For each item, please circle if it would stop you from attending.”
Each survey participant received an incentive of $20 upon completion. The study was approved by the University’s Institutional Review Board and conducted after receiving informed consent from the participants.
A researcher-developed survey was used to collect demographic information and participants also completed the OES. The OES is a 4-point Likert-type scale with responses as follows: 1 = Definitely No; 2 = Probably No; 3 = Probably Yes; and 4 = Definitely Yes. Four subscales were identified by Dumas and colleagues, which were intervention demands (four items), personal or family stressors or obstacles (four items), relevance of or trust in intervention (four items), and time and scheduling demands (two items) (20, 23).
Exploratory factor analysis (EFA) was used to determine the factor structure of the OES in a population of low-income African American parents for comparison to the original factor structure and that previously reported in primarily Caucasian sample of parents. EFA has been conducted to validate the OES in other populations (24), and therefore, the current study extends this methodology to low-income African Americans. An EFA examines correlations among individual measure items to identify the underlying latent factors or instrument subscales. This procedure controls measurement error to identify latent factors underlying the manifest variables rather than simply condensing information provided by the manifest variables, which is typically seen in principal component analysis. The EFA used maximum likelihood estimation and oblique quartimin rotation. Factor loadings greater than 0.40 were considered significant and the squared factor loading represents the amount of variance shared between that item and the factor (or the amount of variance explained by that factor). The criterion was set to minimize the number of non-trivial cross-loadings and enhance interpretability. In addition, parallel analysis and Velicer’s minimum average partial test were used as validating procedures. If these more sophisticated techniques suggested differing factors, factors from these techniques would be reported as a replacement for the factors reported from the more traditional techniques discussed above. Fortunately, results from the traditional methods held consistent, and results from the more sophisticated techniques are not reported for continuity and to reduce possible confusion.
The EFA identified a three-factor solution on the items of the OES in a sample of low-income African Americans. The Chi-squared test demonstrated that there was not enough evidence to reject the hypothesis that the three factors were sufficient (χ2 = 14.32, p < 0.001), thus, supporting a stable factor structure. The Tucker and Lewis’ Reliability Coefficient was 0.97, exceeding the 0.94 cutoff for close fit and the root mean square error was 0.03, exceeding the 0.05 cutoff for close fit. The three factors explained 64.6% of the variance, see Table 2. The three factors’ eigenvalues were 6.19, 1.15, and 1.06, respectively (Table 2). The mean scores on the three factors were: 1.4 (SD = 0.5) for Factor 1, 1.6 (SD = 0.6) for Factor 2, and 2.0 (SD = 0.7) for Factor 3. All items, except one loaded on a factor, meaning all items were appropriate (see Table 3 for complete factor loadings). The total scale had moderate reliability (Fleiss’ Kappa = 0.64) and had good internal consistency (Cronbach’s alpha = 0.74). Cronbach’s alpha values for individual factors ranged from 0.50 to 0.87 (Table 4).
The current study supports the necessity for evaluating the psychometric properties of study instruments for use with specific populations. Data from a sample of low-income African American parents living in an urban location in the southern United States differed from the original factor structure of a sample of adults that was economically and ethnically diverse and from a predominately Caucasian sample of parents from the Midwestern United States, which was analyzed for differences by educational group. Wilson et al. (24) found that the data from their more educated sample was similar to the original factor solution, but parents who had a high school education or less differed in their factor analysis. Similar to the current sample, they found a 3-factor solution rather than the original 4-factor solution. However, the individual items across the two samples loaded differently from each other and from the original sample (See Table 5 for a summary of all studies). Although more data are needed to further disentangle the effects of educational and poverty status from race and ethnicity, these data suggest that race and ethnicity have an effect above that of education. Larger samples with greater representation of parents who are African American, but not poor, in addition to those who are poor are needed to further understand for whom this instrument should be used.
Further instrument development research is needed. New items may need to be added or current items modified and evaluated in this population as well as other population to fully understand the barriers for help-seeking for both health promotion and prevention and for treatment of child health and development problems. As new items are developed, future studies may include both EFA as well as confirmatory factor analysis. Although the factor analysis identified a 3-factor solution, the items did not always fit together conceptually. These data clearly suggest that validation studies are needed when using instruments that were developed for use with divergent populations. Understanding the barriers for help-seeking is necessary to tailor messages and programs for the needs of various individuals and groups.
In addition to examining the psychometric properties of instruments for use in diverse populations, especially those with limited educational levels, instruments should be evaluated for reading level and cultural sensitivity. Over 90 million American adults are either functionally illiterate or have insufficient literacy skills (25). Relatedly, beyond limited general literacy skills, 14% of adults have insufficient health literacy skills and another 22% have only the very minimum adequacy (26). Literacy levels may affect the accuracy with which individuals may answer survey questions and, more importantly, low health literacy has been associated with various health behaviors and health outcomes (27–32), especially for those individuals who are most at risk for adverse outcomes such as racial/ethnic minorities. It has been shown that materials commonly used in clinical settings may not be suitable for those with limited literacy skills, especially health literacy (17, 33, 34). Health professional have begun to understand the importance of health literacy in patient and parent education and efforts are being made to develop and implement policies and practices to ensure better patient understanding of health information. Researchers must also make sure that the instruments they use in research are at the appropriate reading level and are culturally sensitive to ensure that the instruments are measuring what they intend to measure. Care must be taken with using instruments in diverse samples without evaluating the psychometric properties. Additionally, alternate methods of instrument administration such as reading the questions to the participant may be used. However, empirical data are needed to determine if oral administration is acceptable to the participant and if it improves participant understanding. In some cases, visual aids such as pictures and diagrams may ease the literacy burden for participants with limited literacy skills.
Although the data suggest that the instrument may need to be revised for use with low-income African American parents, there were some limitations to our study. The sample was a convenience sample recruited from the waiting room of a pediatric primary care office in an urban, academic, medical center in the southern United States. While the sample was representative of our clinic population, our sample and, thus, our findings, may differ from samples collected in other locations. Having only one local data collection site is a limitation. More data are needed to evaluate the usefulness of the OES in other samples. An additional limitation is that we did not have a sufficient number of participants from other races/ethnicities to do any comparisons between diverse groups nor did we have individuals who were African American with higher levels of income and/or education. Future research is needed to more fully evaluate the instrument for use across diverse groups.
In conclusion, prior to using the OES in future studies with populations that differ from the original sample, more research is needed. Additional items should be added and current items may need to be revised to reduce the literacy burden and improve cultural sensitivity. The psychometric properties should be evaluated again in another sample of low-income, African American parents. Psychometric properties should be assessed in other populations as well prior to use. Lastly, one the instrument demonstrates adequate conceptual and statistical quality; studies are needed to determine it’s usefulness in predicting appropriate outcome variables.
Conflict of Interest Statement
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The study was funded by a grant from the Pilot Research Grant Program, University of Louisville, School of Medicine, Department of Pediatrics; Louisville, Kentucky.
2. Brooks-Gunn J, Linver MR, Fauth RC. Children’s competence and socioeconomic status in the family and neighborhood. In: Elliot AJ, Dweck CS editors. Handbook of competence and motivation. New York: Guilford Press (2005). p. 414–35.
3. Day C. Parenting children in high-risk environments: An examination of maternal sensitivity in poverty. In: Davis DW, Logsdon MC editors. Maternal sensitivity: A foundation for clinical practice. New York, NY: Nova Science Publishers, Inc (2011). p. 195–214.
4. Hanson MJ, Miller AD, Diamond K, Odom S, Lieber J, Butera G, et al. Neighborhood community risk influences on preschool children’s development and school readiness. Infants Young Children (2011) 24(1):87–100. doi: 10.1097/IYC.0b013e3182008dd0
5. Lee L-C, Huang K-Y, Halpern CT, Newschaffer CJ. The impact of maternal depression on developmental psychopathology during early childhood. In: Columbus AM editor. Advances in psychology research. (Vol. 49), Hauppauge, NY: Nova Science Publishers (2007). p. 81–106.
10. Dumas JE, Nissley J, Nordstrom A, Smith EP, Prinz RJ, Levine DW. Home chaos: Sociodemographic, parenting, interactional, and child correlates. J Clin Child Adolesc Psychol (2005) 34:93–104. doi:10.1207/s15374424jccp3401_9
13. Brooks-Gunn J, Rouse CE, McLanahan S. Racial and ethnic gaps in school readiness. In: Pianta RC, Cox MJ, Snow KL editors. School readiness & the transition to kindergarten in the era of accountability. Baltimore, MD: Brookes (2007). p. 283–306.
14. Cabrera NJ, Fagan J, Wight V, Schadler C. Influence of mother, father, and child risk on parenting and children’s cognitive and social behaviors. Child Dev (2011) 82(6):1985–2005. doi:10.1111/j.1467-8624.2011.01667.x
15. Committee on Psychosocial Aspects of Child and Family Health, Committee on Early Childhood Adoption and Dependent Care, Section on Developmental and Behavioral Pediatrics, Garner AS, Shonkoff JP, Siegel BS, et al. Early childhood adversity, toxic stress, and the role of the pediatrician: Translating developmental science into lifelong health. Pediatrics (2012) 129(1):e224–31. doi:10.1542/peds.2011-2662
16. Crosnoe R, Leventhal T, Wirth R, Pierce KM, Pianta RC. Family socioeconomic status and consistent environmental stimulation in early childhood. Child Dev (2010) 81:972–87. doi:10.1111/j.1467-8624.2010.01446.x
17. Davis DW, Jones VF, Logsdon MC, Ryan L, Wilkerson-McMahon M. Health promotion in pediatric primary care: Importance of health literacy and communication practices. Clin Pediatr (2013) 52(12):1127–34. doi:10.1177/0009922813506607
19. Webster-Stratton C, Reid M. Treating conduct problems and strengthening social and emotional competence in young children: The dina dinosaur treatment program. J Emot Behav Disord (2003) 11(3):130–43. doi:10.1177/10634266030110030101
20. Dumas JE, Nissley-Tsiopinis J, Moreland AD. From intent to enrollment, attendance, and participation in preventive parenting groups. J Child Fam Stud (2007) 16(1):1–26. doi:10.1007/s10826-006-9042-0
21. Perrin EC, Sheldrick R, McMenamy JM, Henson BS, Carter AS. Improving parenting skills for families of young children in pediatric settings: A randomized clinical trial. JAMA Pediatr (2014) 168(1):16–24. doi:10.1001/jamapediatrics.2013.2919
22. Thornton S, Calam R. Predicting intention to attend and actual attendance at a universal parent-training programme: A comparison of social cognition models. Clin Child Psychol Psychiatry (2011) 16(3):365–83. doi:10.1177/1359104510366278
24. Wilson SM, Wildman BG, Ciesla J, Smith AW, Dempster R. Factor structure of the Obstacles to Engagement Scale: Problems, solutions, and hypotheses. J Child Fam Stud (2014). doi:10.1007/s10826-014-9899-2
25. Kirsch I, Jungeblut A, Jenkins L, Kolstad A. Adult literacy in America: A first look at the findings of the National Adult Literacy Survey. 3rd ed. Washington, DC: U.S. Department of Education, National Center for Education (2002).
26. Kutner M, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics, publication no. 2006-483 (2006).
28. De Walt DA, Dilling MH, Rosenthal MS, Pignone MP. Low parental literacy is associated with worse asthma care measures in children. Ambulat Pediatr (2007) 7(1):25–31. doi:10.1016/j.ambp.2006.10.001
30. Omachi TA, Sarkar U, Yelin EH, Blanc PD, Katz PP. Lower health literacy is associated with poorer health status and outcomes in chronic obstructive pulmonary disease. J Gen Intern Med (2013) 28(1):74–81. doi:10.1007/s11606-012-2177-3
Keywords: barriers to participation, engagement, Parents, low-income, African Americans
Citation: Davis DW, Dempster R, Myers JA, Jones VF, Ryan L and Logsdon MC (2014) Evaluation of the factor structure of the Obstacles to Engagement Scale with low-income African American parents. Front. Pediatr. 2:139. doi: 10.3389/fped.2014.00139
Received: 10 October 2014; Accepted: 25 November 2014;
Published online: 17 December 2014.
Edited by:Marie Leiner, Texas Tech University Health Sciences Center, USA
Reviewed by:Mercedes Ellis Ramirez, Texas Tech University Health Sciences Center, USA
Jesus Peinado, Texas Tech University Health Sciences Center, USA
Laura Nabors, University of Cincinnati, USA
Neal Krause, University of Michigan, USA
Copyright: © 2014 Davis, Dempster, Myers, Jones, Ryan and Logsdon. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Deborah Winders Davis, Department of Pediatrics, University of Louisville, 571 S. Floyd Street, Suite 412, Louisville, KY 40202, USA e-mail: firstname.lastname@example.org