Evaluation of the factor structure of the Obstacles to Engagement Scale with low-income African American parents

Introduction

Early childhood is a critical period for laying the foundation for later child health, behavior, and development (1–8). The quality of parent–child interactions during early childhood has important implications for long-term health, development, and behavioral outcomes and for health and educational resource utilization (1, 9–11). Risk factors for less than optimal child development include poverty, minority status, and low levels of parent education (1, 12, 13). There are many reasons why low-income, ethnic minority families may need additional support in establishing positive parenting skills, including positive parent-child interactions, such as lack of knowledge about health and development, lower health literacy, higher levels of stress, and fewer available resources (3, 14–18). More data are needed to understand what specific factors serve as barriers for parents in receiving the support they need.

Parenting programs have been shown to be successful in improving child outcomes. With early intervention and prevention, many negative consequences can be avoided and further psychosocial dysfunction can be prevented (19). However, much remains unknown about the factors that serve as barriers for parents’ help-seeking to maximize child health and development and to prevent adverse outcomes. Parental attendance, in general, has been low for parenting programs even when their children have documented or perceived behavior problems (20–22). The Obstacle to Engagement Scale (OES) has been used in studies to understand help-seeking behavior of parents for their children’s behavior (20, 23). Data obtained while using the tool has potential to inform the development of interventions aimed at engaging parents and removing real or perceived barriers for their participation in programs that promote the development of positive parenting skills. However, care must be taken in using such instruments that have been developed and validated for other populations without validating them for other populations of interest.

Understanding barriers to parental help-seeking may lead to the development of targeted messages to increase service utilization and reduce health disparities in children who are at greatest risk such as those from low-income families and ethnic minorities. Barriers may differ with parents who differ in socioeconomic and/or educational status, race/ethnicity, age, gender, and health status, to name a few. OES (20) is a measure of potential barriers to treatment that was specifically developed to assess barriers to participation in preventative behavioral parenting classes for parents of preschool-age children. The original sample was described as economically and ethnically diverse (20) with just over half self-identifying as African American. The sample was recruited from day care centers in two metropolitan cities in the United States (N = 410). Others have validated the scale in samples that were predominately Caucasian (24) who were recruited from pediatric primary care settings in the Midwestern United States. To date, no studies have been found that specifically examine the psychometric properties of the scale when used with low-income African American parents. Therefore, the current study aims to determine if the factor structures previously described in other groups are the same as that found when the OES is used with a sample of low-income African American parents. Administration of the questionnaire asked parents about reasons that might prohibit their attendance at preventative parenting classes.

Materials and Methods

Sample

As part of a larger study, a sample of 190 parents or legal guardians completed a survey in the waiting room of a primary care pediatric academic practice that serves predominantly low-income families in an urban location in the southern United States of America. Data collection took place May through August 2011. All primary caregivers with children aged 3–8 years were invited to participate. For the larger study, parents were asked about their preferences for receiving parenting information and about issues related to stigma in addition to their obstacles to engagement. For the current study we focused on the African American participants who had complete data from the OES (N = 114). See Table 1 for demographic information on participants.

TABLE 1

Table 1. Demographic information of participants (N = 114).

Procedures

Parents completed a survey containing demographic characteristics of themselves and their children. Additionally, parents also completed the OES (barriers to treatment) (20). The instructions for the OES portion of the survey were as follows. “There are many reasons why parents might choose to attend or not to attend parenting classes. Listed below are some reasons that might keep parents from attending. For each item, please circle if it would stop you from attending.”

Each survey participant received an incentive of $20 upon completion. The study was approved by the University’s Institutional Review Board and conducted after receiving informed consent from the participants.

Instruments

A researcher-developed survey was used to collect demographic information and participants also completed the OES. The OES is a 4-point Likert-type scale with responses as follows: 1 = Definitely No; 2 = Probably No; 3 = Probably Yes; and 4 = Definitely Yes. Four subscales were identified by Dumas and colleagues, which were intervention demands (four items), personal or family stressors or obstacles (four items), relevance of or trust in intervention (four items), and time and scheduling demands (two items) (20, 23).

Statistical Analysis

Exploratory factor analysis (EFA) was used to determine the factor structure of the OES in a population of low-income African American parents for comparison to the original factor structure and that previously reported in primarily Caucasian sample of parents. EFA has been conducted to validate the OES in other populations (24), and therefore, the current study extends this methodology to low-income African Americans. An EFA examines correlations among individual measure items to identify the underlying latent factors or instrument subscales. This procedure controls measurement error to identify latent factors underlying the manifest variables rather than simply condensing information provided by the manifest variables, which is typically seen in principal component analysis. The EFA used maximum likelihood estimation and oblique quartimin rotation. Factor loadings greater than 0.40 were considered significant and the squared factor loading represents the amount of variance shared between that item and the factor (or the amount of variance explained by that factor). The criterion was set to minimize the number of non-trivial cross-loadings and enhance interpretability. In addition, parallel analysis and Velicer’s minimum average partial test were used as validating procedures. If these more sophisticated techniques suggested differing factors, factors from these techniques would be reported as a replacement for the factors reported from the more traditional techniques discussed above. Fortunately, results from the traditional methods held consistent, and results from the more sophisticated techniques are not reported for continuity and to reduce possible confusion.

Results

The EFA identified a three-factor solution on the items of the OES in a sample of low-income African Americans. The Chi-squared test demonstrated that there was not enough evidence to reject the hypothesis that the three factors were sufficient (χ² = 14.32, p < 0.001), thus, supporting a stable factor structure. The Tucker and Lewis’ Reliability Coefficient was 0.97, exceeding the 0.94 cutoff for close fit and the root mean square error was 0.03, exceeding the 0.05 cutoff for close fit. The three factors explained 64.6% of the variance, see Table 2. The three factors’ eigenvalues were 6.19, 1.15, and 1.06, respectively (Table 2). The mean scores on the three factors were: 1.4 (SD = 0.5) for Factor 1, 1.6 (SD = 0.6) for Factor 2, and 2.0 (SD = 0.7) for Factor 3. All items, except one loaded on a factor, meaning all items were appropriate (see Table 3 for complete factor loadings). The total scale had moderate reliability (Fleiss’ Kappa = 0.64) and had good internal consistency (Cronbach’s alpha = 0.74). Cronbach’s alpha values for individual factors ranged from 0.50 to 0.87 (Table 4).

TABLE 2

Table 2. Results from the factor analysis.

TABLE 3

Table 3. Factor loadings.

TABLE 4

Table 4. Cronbach’s Alpha scores.

Discussion

The current study supports the necessity for evaluating the psychometric properties of study instruments for use with specific populations. Data from a sample of low-income African American parents living in an urban location in the southern United States differed from the original factor structure of a sample of adults that was economically and ethnically diverse and from a predominately Caucasian sample of parents from the Midwestern United States, which was analyzed for differences by educational group. Wilson et al. (24) found that the data from their more educated sample was similar to the original factor solution, but parents who had a high school education or less differed in their factor analysis. Similar to the current sample, they found a 3-factor solution rather than the original 4-factor solution. However, the individual items across the two samples loaded differently from each other and from the original sample (See Table 5 for a summary of all studies). Although more data are needed to further disentangle the effects of educational and poverty status from race and ethnicity, these data suggest that race and ethnicity have an effect above that of education. Larger samples with greater representation of parents who are African American, but not poor, in addition to those who are poor are needed to further understand for whom this instrument should be used.

TABLE 5

Table 5. Factors loadings for Obstacles to Engagement Scales in four separate samples.

Further instrument development research is needed. New items may need to be added or current items modified and evaluated in this population as well as other population to fully understand the barriers for help-seeking for both health promotion and prevention and for treatment of child health and development problems. As new items are developed, future studies may include both EFA as well as confirmatory factor analysis. Although the factor analysis identified a 3-factor solution, the items did not always fit together conceptually. These data clearly suggest that validation studies are needed when using instruments that were developed for use with divergent populations. Understanding the barriers for help-seeking is necessary to tailor messages and programs for the needs of various individuals and groups.

In addition to examining the psychometric properties of instruments for use in diverse populations, especially those with limited educational levels, instruments should be evaluated for reading level and cultural sensitivity. Over 90 million American adults are either functionally illiterate or have insufficient literacy skills (25). Relatedly, beyond limited general literacy skills, 14% of adults have insufficient health literacy skills and another 22% have only the very minimum adequacy (26). Literacy levels may affect the accuracy with which individuals may answer survey questions and, more importantly, low health literacy has been associated with various health behaviors and health outcomes (27–32), especially for those individuals who are most at risk for adverse outcomes such as racial/ethnic minorities. It has been shown that materials commonly used in clinical settings may not be suitable for those with limited literacy skills, especially health literacy (17, 33, 34). Health professional have begun to understand the importance of health literacy in patient and parent education and efforts are being made to develop and implement policies and practices to ensure better patient understanding of health information. Researchers must also make sure that the instruments they use in research are at the appropriate reading level and are culturally sensitive to ensure that the instruments are measuring what they intend to measure. Care must be taken with using instruments in diverse samples without evaluating the psychometric properties. Additionally, alternate methods of instrument administration such as reading the questions to the participant may be used. However, empirical data are needed to determine if oral administration is acceptable to the participant and if it improves participant understanding. In some cases, visual aids such as pictures and diagrams may ease the literacy burden for participants with limited literacy skills.

Although the data suggest that the instrument may need to be revised for use with low-income African American parents, there were some limitations to our study. The sample was a convenience sample recruited from the waiting room of a pediatric primary care office in an urban, academic, medical center in the southern United States. While the sample was representative of our clinic population, our sample and, thus, our findings, may differ from samples collected in other locations. Having only one local data collection site is a limitation. More data are needed to evaluate the usefulness of the OES in other samples. An additional limitation is that we did not have a sufficient number of participants from other races/ethnicities to do any comparisons between diverse groups nor did we have individuals who were African American with higher levels of income and/or education. Future research is needed to more fully evaluate the instrument for use across diverse groups.

In conclusion, prior to using the OES in future studies with populations that differ from the original sample, more research is needed. Additional items should be added and current items may need to be revised to reduce the literacy burden and improve cultural sensitivity. The psychometric properties should be evaluated again in another sample of low-income, African American parents. Psychometric properties should be assessed in other populations as well prior to use. Lastly, one the instrument demonstrates adequate conceptual and statistical quality; studies are needed to determine it’s usefulness in predicting appropriate outcome variables.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The study was funded by a grant from the Pilot Research Grant Program, University of Louisville, School of Medicine, Department of Pediatrics; Louisville, Kentucky.

References

1. Bornstein MH, Bradley RH. Socioeconomic status, parenting, and child development. Mahwah, NJ: Lawrence Erlbaum Associates Publishers (2003).

Google Scholar

2. Brooks-Gunn J, Linver MR, Fauth RC. Children’s competence and socioeconomic status in the family and neighborhood. In: Elliot AJ, Dweck CS editors. Handbook of competence and motivation. New York: Guilford Press (2005). p. 414–35.

Google Scholar

3. Day C. Parenting children in high-risk environments: An examination of maternal sensitivity in poverty. In: Davis DW, Logsdon MC editors. Maternal sensitivity: A foundation for clinical practice. New York, NY: Nova Science Publishers, Inc (2011). p. 195–214.

Google Scholar

4. Hanson MJ, Miller AD, Diamond K, Odom S, Lieber J, Butera G, et al. Neighborhood community risk influences on preschool children’s development and school readiness. Infants Young Children (2011) 24(1):87–100. doi: 10.1097/IYC.0b013e3182008dd0

CrossRef Full Text | Google Scholar

5. Lee L-C, Huang K-Y, Halpern CT, Newschaffer CJ. The impact of maternal depression on developmental psychopathology during early childhood. In: Columbus AM editor. Advances in psychology research. (Vol. 49), Hauppauge, NY: Nova Science Publishers (2007). p. 81–106.

Google Scholar

6. Ramey CT. Pesistant effects of early childhood education on high risk children and their mothers. Appl Dev Sci (2000) 4(1):2–14. doi: 10.1207/S1532480XADS0401_1

CrossRef Full Text | Google Scholar

7. Ramey CT, Ramey SL. Early learning and school readiness: Can early intervention make a difference? Merrill Palmer Q (2004) 50:471–91. doi:10.1353/mpq.2004.0034

CrossRef Full Text | Google Scholar

8. Sameroff A. A unified theory of development: A dialectic integration of nature and nurture. Child Dev (2010) 81(1):6–22. doi:10.1111/j.1467-8624.2009.01378.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

9. Bradley RH. Environment and parenting. Mahwah, NJ: Lawrence Erlbaum Associates Publishers (2002).

Google Scholar

10. Dumas JE, Nissley J, Nordstrom A, Smith EP, Prinz RJ, Levine DW. Home chaos: Sociodemographic, parenting, interactional, and child correlates. J Clin Child Adolesc Psychol (2005) 34:93–104. doi:10.1207/s15374424jccp3401_9

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

11. Sameroff A. The transactional model of development: How children and contexts shape each other. Washington, DC: American Psychological Association (2009).

Google Scholar

12. Bradley RH, Corwyn RF. Socioeconomic status and child development. Annu Rev Psychol (2002) 53:371–99. doi:10.1146/annurev.psych.53.100901.135233

CrossRef Full Text | Google Scholar

13. Brooks-Gunn J, Rouse CE, McLanahan S. Racial and ethnic gaps in school readiness. In: Pianta RC, Cox MJ, Snow KL editors. School readiness & the transition to kindergarten in the era of accountability. Baltimore, MD: Brookes (2007). p. 283–306.

Google Scholar

14. Cabrera NJ, Fagan J, Wight V, Schadler C. Influence of mother, father, and child risk on parenting and children’s cognitive and social behaviors. Child Dev (2011) 82(6):1985–2005. doi:10.1111/j.1467-8624.2011.01667.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

15. Committee on Psychosocial Aspects of Child and Family Health, Committee on Early Childhood Adoption and Dependent Care, Section on Developmental and Behavioral Pediatrics, Garner AS, Shonkoff JP, Siegel BS, et al. Early childhood adversity, toxic stress, and the role of the pediatrician: Translating developmental science into lifelong health. Pediatrics (2012) 129(1):e224–31. doi:10.1542/peds.2011-2662

CrossRef Full Text | Google Scholar

16. Crosnoe R, Leventhal T, Wirth R, Pierce KM, Pianta RC. Family socioeconomic status and consistent environmental stimulation in early childhood. Child Dev (2010) 81:972–87. doi:10.1111/j.1467-8624.2010.01446.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

17. Davis DW, Jones VF, Logsdon MC, Ryan L, Wilkerson-McMahon M. Health promotion in pediatric primary care: Importance of health literacy and communication practices. Clin Pediatr (2013) 52(12):1127–34. doi:10.1177/0009922813506607

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

18. Weekes CV. African Americans and health literacy: A systematic review. ABNF J (2012) 23(4):76–80.

Google Scholar

19. Webster-Stratton C, Reid M. Treating conduct problems and strengthening social and emotional competence in young children: The dina dinosaur treatment program. J Emot Behav Disord (2003) 11(3):130–43. doi:10.1177/10634266030110030101

CrossRef Full Text | Google Scholar

20. Dumas JE, Nissley-Tsiopinis J, Moreland AD. From intent to enrollment, attendance, and participation in preventive parenting groups. J Child Fam Stud (2007) 16(1):1–26. doi:10.1007/s10826-006-9042-0

CrossRef Full Text | Google Scholar

21. Perrin EC, Sheldrick R, McMenamy JM, Henson BS, Carter AS. Improving parenting skills for families of young children in pediatric settings: A randomized clinical trial. JAMA Pediatr (2014) 168(1):16–24. doi:10.1001/jamapediatrics.2013.2919

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

22. Thornton S, Calam R. Predicting intention to attend and actual attendance at a universal parent-training programme: A comparison of social cognition models. Clin Child Psychol Psychiatry (2011) 16(3):365–83. doi:10.1177/1359104510366278

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

23. Dempster R, Wildman B, Keating A. The role of stigma in parental help-seeking for child behavior problems. J Clin Child Adolesc Psychol (2013) 42(1):56–67. doi:10.1080/15374416.2012.700504

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

24. Wilson SM, Wildman BG, Ciesla J, Smith AW, Dempster R. Factor structure of the Obstacles to Engagement Scale: Problems, solutions, and hypotheses. J Child Fam Stud (2014). doi:10.1007/s10826-014-9899-2

CrossRef Full Text | Google Scholar

25. Kirsch I, Jungeblut A, Jenkins L, Kolstad A. Adult literacy in America: A first look at the findings of the National Adult Literacy Survey. 3rd ed. Washington, DC: U.S. Department of Education, National Center for Education (2002).

Google Scholar

26. Kutner M, Greenberg E, Jin Y, Paulsen C. The health literacy of America’s adults: Results from the 2003 National Assessment of Adult Literacy. Washington, DC: National Center for Education Statistics, publication no. 2006-483 (2006).

Google Scholar

27. Baker DW, Parker RM, Williams MV, Clark WS. Health literacy and the risk of hospital admission. J Gen Intern Med (1998) 13(12):791–8. doi:10.1046/j.1525-1497.1998.00242.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

28. De Walt DA, Dilling MH, Rosenthal MS, Pignone MP. Low parental literacy is associated with worse asthma care measures in children. Ambulat Pediatr (2007) 7(1):25–31. doi:10.1016/j.ambp.2006.10.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

29. James BD, Boyle PA, Bennett JS, Bennett DA. The impact of health and financial literacy on decision making in community-based older adults. Gerontology (2012) 58(6):531–9. doi:10.1159/000339094

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

30. Omachi TA, Sarkar U, Yelin EH, Blanc PD, Katz PP. Lower health literacy is associated with poorer health status and outcomes in chronic obstructive pulmonary disease. J Gen Intern Med (2013) 28(1):74–81. doi:10.1007/s11606-012-2177-3

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

31. Weiss BD. 20 common problems in primary care. New York, NY: McGraw-Hill (1999).

Google Scholar

32. Weiss BD. Health literacy and patient safety. Chicago, IL: AMA Foundation (2007).

Google Scholar

33. Freda MC. The readability of American Academy of Pediatrics patient education brochures. J Pediatr Health Care (2005) 19(3):151–6. doi:10.1016/j.pedhc.2005.01.013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

34. Ryan L, Logsdon MC, McGill S, Stikes R, Senior B, Helinger B, et al. Evaluation of printed health education materials for use by low-education families. J Nurs Scholarship (2014) 46(4):218–28. doi:10.1111/jnu.12076

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar