Incremental Validity and Informant Effect from a Multi-Method Perspective: Assessing Relations between Parental Acceptance and Children’s Behavioral Problems

This study examines the relationships between perceived parental acceptance and children’s behavioral problems (externalizing and internalizing) from a multi-informant perspective. Using mothers, fathers, and children as sources of information, we explore the informant effect and incremental validity. The sample was composed of 681 participants (227 children, 227 fathers, and 227 mothers). Children’s (40% boys) ages ranged from 9 to 17 years (M = 12.52, SD = 1.81). Parents and children completed both the Parental Acceptance Rejection/Control Questionnaire (PARQ/Control) and the check list of the Achenbach System of Empirically Based Assessment (ASEBA). Statistical analyses were based on the correlated uniqueness multitrait-multimethod matrix (model MTMM) by structural equations and different hierarchical regression analyses. Results showed a significant informant effect and a different incremental validity related to which combination of sources was considered. A multi-informant perspective rather than a single one increased the predictive value. Our results suggest that mother–father or child–father combinations seem to be the best way to optimize the multi-informant method in order to predict children’s behavioral problems based on perceived parental acceptance.

Introduction

The progress of psychology is inextricably linked to the development of new and more refined methods and strategies for measuring psychological concepts, models, and intervention programs (Eid and Diener, 2006). A multi-informant approach offers insights into scientific phenomena and can contribute to confirming psychological theories in a way that a single-informant approach cannot. Due to the complexity of constructs evaluated and developmental factors that take place in children’s psychological adjustment, their assessment is mainly multimodal (e.g., rating scales, interviews, and observations), multi-informant (e.g., child, parents, teachers, and mates), and/or multi-trait (Eyde et al., 1993; Ollendick and Hersen, 1993; Mash and Terdal, 1997; Duhig et al., 2000; Meyer et al., 2001; Johnston and Murray, 2003; Achenbach, 2006; Hunsley and Mash, 2007). Specifically for informant assessment, the most reliable source of information on a target’s psychological characteristics is not to be found in his or her self-ratings, nor it is guaranteed by single informant ratings; rather, it is found in the combination of the judgments from the community of the target’s knowledgeable informants. According to this, the multi-informant assessment is mostly accepted by the psychological assessment community as an adequate and useful procedure, since rarely is a unique measure sufficient for providing all the required information needed to form an accurate judgment (Meyer and Archer, 2001; Garb, 2003; De Los Reyes and Kazdin, 2004; Carrasco et al., 2008; Hughes and Gullone, 2010). However, informant effects represent bias that can derive from the use of the same source of information in the assessment of different traits, the knowledge of informants, the observability of assessed traits, the judgment of informants, or the social desirability, among other factors (Cheng and Furnham, 2004; Neyer, 2006). For these reasons, determining the extent to which an informant effect is affecting the assessment of constructs and its relations is an important goal in determining the real construct validity. Individual reports often yield inconsistent data and discrepancies that can create considerable uncertainties in designing interventions and drawing conclusions from research (Klein, 1991; Epkins, 1993; Jané et al., 2000; De Los Reyes and Kazdin, 2004, 2005, 2006; Achenbach, 2006; Goodman et al., 2010; De Los Reyes et al., 2015). For instance, associations between constructs tend to be largest: (a) when a single informant is used, because of shared method variance (Neyer, 2006); (b), when the assessment of interventions has a large effect on parent reports vs. observed child behaviors of children’s externalizing problems (Tarver et al., 2014); or (c) when family members experience their interaction differently and therefore have dissimilar views on parenting and parent child relations (e.g., Lanz et al., 2001; Hoeve et al., 2009). A key reason for these uncertainties originates from the near-exclusive focus on mental health research as applied to whether informant discrepancies reflect measurement error or reporting biases (e.g., Richters, 1992; De Los Reyes, 2011). Consequently, what remains unclear is whether a multi-informant approach to assessment validly captures contextual variations displayed in children’s behavioral problems or whether it instead reflects different perceptions or beliefs about what a symptom is, and, finally, which informants ought to be included in assessments of children and adolescents.

Regarding this last point, another important issue from a multi-informant approach is the differential contribution of a particular source of information in relation to others. That is, the incremental validity or degree to which adding a new informant to the assessment consistently increases the predictive power and decision making (Garb, 2003; Hunsley, 2003; Hunsley and Mash, 2005). Unfortunately, the incremental validity inherent in using and combining multiple assessment methods has not undergone wide empirical testing in the literature on either adult or child assessment (Mash and Terdal, 1997; Hunsley, 2002). Thus, strong psychometric properties of the individual measures are necessary but do not provide sufficient conditions to ensure the incremental validity of incorporating these measures into the assessment process. Furthermore, not only is the research that deals directly with incremental validity in child assessment relatively small, the incremental validity of mothers’ vs. fathers’ reports has seldom been tested (Johnston and Murray, 2003).

With regard to cross-informant use, some studies support the incremental value of adults’ over children’s information when externalizing problems are measured (Loeber et al., 1991; Carrasco et al., 2008). However, the use of adults’ information in children’s assessment does not always augment the value of using only one source of information (Biederman et al., 1990). On the other hand, for older children, when assessing internalizing problems or covert behaviors, there is some evidence for the incremental value of youth self-reports over parents reports (Langhinrichsen et al., 1990; Cantwell et al., 1997; Johnston and Murray, 2003).

One of the most consistent observations in the field of child assessment is the correspondence levels between informants’ reports, which range from low to moderate in magnitude (Achenbach et al., 1987; Duhig et al., 2000; Achenbach, 2011; Markon et al., 2011; De Los Reyes et al., 2015). The evidence usually shows that pairs of informants who observed children in the same context (e.g., pairs of parents or pairs of teachers) tend to show greater levels of correspondence than pairs of informants who observed children in different contexts (e.g., parent and teacher). Accordingly, some studies have found that the cross-informant agreement was moderate to high between mother and father, and moderate to low between father–child and mother–child pairs (Grigorenko et al., 2010; Weitkamp et al., 2013). Correspondence between mothers and children tend to be higher than correspondence between fathers and children (Grigorenko et al., 2010) and mother–child reports tend to find a greater endorsement than father–child reports (Lapouse and Monk, 1958; Achenbach et al., 1987; Stanger and Lewis, 1993; De Los Reyes et al., 2015). Also, the confluence of informants’ reports about children’s externalizing problems (e.g., aggression and hyperactivity concerns) tends to be higher than that concerning internalizing problems (e.g., anxiety and depression). In this regard, maternal and paternal reports show moderate correspondence when rating internalizing behavior problems in children and a larger correspondence in ratings of externalizing behavior problems in children (Achenbach et al., 1987; Duhig et al., 2000; Grigorenko et al., 2010). This evidence may reflect the greater correspondence between reports of directly observable behaviors than internalized behaviors. There is also evidence supporting claims that the degree of acquaintance between parents and children is a factor that leads to different parental ratings (Hughes and Gullone, 2010). The variability of correspondence found between the different pairs of informants is probably reflective of both the potential informant effect and the differential contribution of each source of information to the assessment’s target. Furthermore, we would like to remark that the variation of the responses will be due to real differences from individual subjects, and the variation of the subjects on the variable won’t be a continuous uniform distribution, but its favorable or unfavorable position on the studied object will be according to their perception (Likert, 1932).

This study tries to explore from a multi-informant approach the relations between parental acceptance and children’s internalizing and externalizing problems. Perceived parental acceptance is one of the main factors involved in children’s psychological adjustment, as is shown from the interpersonal acceptance-rejection theory (IPARTheory; Rohner, 1986; Rohner et al., 2012). Parental rejection (the opposite of parental acceptance) implies the absence or a significant withdrawal of parental warmth, affection, care, comfort, concern, nurturance, support, or love, and the presence of a variety of physically and psychologically hurtful behaviors and effects (Rohner and Khaleque, 2005; Rohner et al., 2012). Meta-analysis studies on this subject have found that rejection has consistently negative effects on the psychological adjustment and behavioral functioning of both children and adults worldwide (Khaleque and Rohner, 2002; Rohner and Khaleque, 2005; Rohner et al., 2012). The same body of research also shows that children who perceive their parents as being rejecting tend to experience distress, and in turn develop a specific cluster of internalizing (i.e., emotional instability, depression) and externalizing (i.e., aggression, delinquency) problems (McLeod et al., 2007; Hoeve et al., 2009; Rohner and Khaleque, 2010; Khaleque and Rohner, 2012; Khaleque, 2015; Ramírez-Lucas et al., 2015). However, no studies from this perspective have been conducted, to our knowledge, that explore either the informant effect or the incremental validity of parents’ and children’s perceived parental acceptance on externalizing and internalizing behavioral problems. Accordingly, no specific results are expected and no particular hypotheses are going to be tested. The first aim of this study is to test for evidence of informant effects related to the links between parental acceptance and children’s behavioral problems as measured by children, fathers, and mothers through a round-robin design, in which all informants rate all targets. The second aim is to explore the incremental validity of the informants. Specifically, we deal with two questions: (1) Are there significant informant effects predicting children’s behavioral problems based on perceived parental acceptance? (2) What is the incremental validity of the children’s perceived parental acceptance over the parent’s perceived parental acceptance in predicting the children’s behavioral problems?

Materials and Methods

Sample

The sample was composed of 681 participants (227 children, 227 fathers, and 227 mothers). Children’s (40% boys; n = 90) ages ranged from 9 to 17 years (M = 12.52, SD = 1.81): 37% (n = 61) were between 9 and11 years, 47% (n = 107) were between 12 and 13 years, 20% (n = 46) were between 14 and 15, and 6% (n = 13) were between 16 and 17 years.

All of the children attended school, the majority lived in two-parent households (91%), and the mean number of siblings was three. Of the parents, 88% of fathers and 70% of mothers were employed. Occupational titles for mothers and fathers (respectively) were: major professionals (17 and 17%), lesser professionals (40 and 33%), semi-skilled workers (18 and 26%), and unskilled workers (25 and 24%). The mothers’ and fathers’ education levels were: university studies (40 and 35%), high school studies (40 and 57%), and primary studies (20 and 8%).

This sample is part of a larger sample of a general study about parental acceptance and children’s psychological adjustment in the Spanish population. Children were selected according to mother–father–child matched participation. This sample represents 22% of the total sample (N = 1036). The total sample was randomly selected from public schools and publically funded private schools in different cities and communities of Spain. The participation rate of the total families was 91.5%.

No significant differences were found between participant and non-participant families in the demographic variables (i.e., child’s sex, age, and socioeconomic level).

Measures

All measures were filled in by children, mothers, and fathers using the appropriate versions of the instruments described below.

Parental Acceptance

Four versions of the Parental Acceptance-Rejection/Control Questionnaire were used to report on perceived parental acceptance, two for children (mother and father versions, one to report about each parent) and two for parents (one version for mothers and another version for fathers). Children filled in both mother and father versions (Parental Acceptance-Rejection/Control Questionnaire, Child PARQ/Control: mother-short version for children and Child PARQ/Control: father -short version for children). Mothers filled in mother versions and fathers filled in father versions (Parental Acceptance-Rejection/Control Questionnaire, PARQ/Control: Mother- short version for parents and, PARQ/Control: father- short version for parents; Rohner, 1990; Rohner and Khaleque, 2005; Spanish adaptation by Del Barrio et al., 2014). The short versions of the PARQ/Control for children and for parents consist of 29-item. The PARQ/Control for children is a self-reporting questionnaires with four scales measuring warmth/affection [e.g., “My mother (father) says nice things about me”], hostility/aggression [e.g., “My mother (father) gets angry at me easily”], indifference/neglect [e.g., “My mother (father) pays no attention to me”], and undifferentiated rejection [e.g., “My mother (father) does not really love me”], plus a parental control (permissive-strictness) scale built into it. The PARQ/Control for mothers and fathers are self-reports with the same scales as the version for children; the difference with the children version is that items ask about the mother or father her/himself (e.g., “I get angry at my son easily”). The mother and father versions of the PARQ/Control (short forms) are identical, with the exception of the title changing according to which parent is being assessed. In all versions items are scored on a 4-point Likert-type scale ranging from 4 (almost always true) through 1 (almost never true). The sum of the first four scales (24 items) constitutes a measure of overall perceived maternal and paternal acceptance/rejection (with the entire warmth scale reverse scored). A greater score indicates a perception of greater parental rejection. Evidence regarding the validity and reliability of the PARQ/Control has been very well supported (Khaleque and Rohner, 2002; Rohner and Khaleque, 2005). Coefficient alphas for the total score in this sample are 0.88 for fathers and 0.97 for mothers in the children versions; and 0.88 for fathers and 0.88 for mothers in the parent version.

Children’s Behavioral Problems

Two versions from the Achenbach System Evidence Based Assessment (Achenbach and Rescorla, 2007) were used to report on the children’s behavioral problems: one for children (YSR) and one for parents (CBCL). Fathers and mothers inform separately about the children’s behavioral problems on the CBCL version. The Youth Self-Report (YSR) is composed of two parts, the first assessing various psychosocial skills and competences, and the second consisting of a check-list of 112 items assessing a large number of behavioral problems, which are aggregated into two broad dimensions: internalizing (anxiety/depression, withdrawal, somatic complaints) and externalizing (breaking rules, aggressive behavior) problems. The items are scored on a 3-point Likert-type scale with anchors of 0 (not true), 1 (somewhat or sometimes true), and 2 (very true or often true). The Children’s Behavioral Check List (CBCL) is similar to YSR, with the exception of having one item more (113 “Other problems”). For the purpose of this study, we only use the check-lists and the two broad dimensions: externalizing and internalizing behavioral problems.

For this sample, the Cronbach’s alphas were 0.75 for the internalizing scale, and 0.73 for the externalizing scale on the YSR version; 0.79 and 0.78 for the internalizing scale, and 0.80 and 0.77 for the externalizing scale on the father-CBCL and mother-CBCL, respectively.

Procedure

Once the cluster sample of schools was selected, an authorization from the school board and an informed consent form from each child’s responsible guardian were collected. Participation was voluntary. The instruments were administered collectively to each school class group in their own classrooms by research personnel trained for this task.

To explore the potential informant effect, we started with the correlated uniqueness model MTMM (Multitrait-multimethod Matrix; Byrne, 1998). According to the correlated uniqueness model, if the different sources are adding systematic variability to the model, we should find significant correlations between errors of the dependent variables reported by the same informant. At the same time, no matter what the global fit of the model is, a significant increase in the model fit should be noted. Second, we used a different hierarchical regression analysis to determine the magnitude of the incremental validity.

Data was analyzed using LISREL 8.9 and SPSS version 20.0 for Windows (SPSS WIN).

Design and Variables

A round-robin design was employed, in which fathers, mothers, and children separately completed all the instruments used. The independent variables were parental acceptance levels as perceived by children, mothers, and fathers. The dependent variables were children’s externalizing and internalizing problems, reported separately by fathers, mothers, and children.

Results

In Table 1 is included the correlation matrix among the variables used. According to the Multitrait-Multimethod matrix logit, if any informant effect exists the Monosource-Multitrait correlation should be higher than the Multisource-Multitrait one. If we focus on the dependent variables, we observe that the correlation between the internalizing and the externalizing problems informed by children (r_int-ext) is 0.54 (monosource-multitrait). This value is higher than other multisource-multitrait correlations such as r_int-pext = 0.15; r_int-mext = 0.19; r_pint-pext = 0.12; or r_mint-ext = 0.02. These results should take us to think about a possible informant effect. The same pattern is found in other variables. Thus, the correlation intra-informant for the same two variables is higher than the correlation inter-informants.

TABLE 1

TABLE 1. Correlation matrix.

In order to obtain more evidences about the informant effect, we tested two models. In the first one (model 1), all the PARQ measures (PARQP, PARQM, MPARQ, and PPARQ) were predictors of all the criterion variables (INT, EXT, MINT, MEXT, PINT, and PEXT; Figure 1). The second model (model 2), was essentially the same, but included the correlations between the errors of each criterion variable reported by each informant (children, mothers, and fathers; Figure 2). We established that if we observed significant correlations between these errors in the second model, and the fit was improved, then it could be reasonable to think about an informant effect.

FIGURE 1

FIGURE 1. Parental acceptance predicting children’s behavioral problems from a multi-informant method with uncorrelated errors (Model 1). Parqp, paternal acceptance reported by children; parqm, maternal acceptance reported by children; mparq, maternal acceptance reported by mothers; pparq, paternal acceptance reported by fathers; int, internalizing problems reported by children; ext, externalizing problems reported by children; mint, internalizing problems reported by mothers; mext, externalizing problems reported by mothers; pext, externalizing problems reported by fathers; pint, internalizing problems reported by fathers.

FIGURE 2

FIGURE 2. Parental acceptance predicting children’s behavioral problems from a multi-informant method with correlated errors (Model 2). Parqp, paternal acceptance reported by children; parqm, maternal acceptance reported by children; mparq, maternal acceptance reported by mothers; pparq, paternal acceptance reported by fathers; int, internalizing problems reported by children; ext, externalizing problems reported by children; mint, internalizing problems reported by mothers; mext, externalizing problems reported by mothers; pext, externalizing problems reported by fathers; pint, internalizing problems reported by fathers.

The fit indexes obtained for the first model were: χ² = 482.66, df = 21; p = 0.00; CFI = 0.57; RMSEA = 0.30; GFI = 0.35; AGFI = 0.35; GFI = 0.75; RMR = 0.15. For model 2, we obtained: χ² = 236.01, df = 18; p = 0.00; CFI = 0.79; RMSEA = 0.22; GFI = 0.86; AGFI = 0.56; RMR = 0.12.

Logically, in terms of fit indexes, both models are not necessarily accepted because we are not looking for a predictive model to explain the relationship between the variables. According to our premise, we should test whether the errors of the various criterion measures from the same informant are correlated. In this sense, model 2 improves the fit of the model 1 (Δχ² = 246.65; Δdf = 3), and the correlations between the errors of the criterion variables reported by the same source of information are significant [e_{int_ext} = 0.43, Critical Ratio (CR) = 9.74; e_{mint_mext} = 0.28, CR = 5.31; e_{pint_pext} = 0.43, CR = 10.69].

These results show a significant effect of the informant. As we can see in Figure 2, children and fathers are the informants that add more variability to the model; that is, the covariance of errors between children’s internalizing and externalizing problems are higher when they are reported by fathers and by children than when they are reported by mothers. In order to quantify the magnitude of the contributions of the various informants, and their incremental validity, we conducted six hierarchical regression analyses.

The results from the hierarchical regression analyses are shown in Table 2. The contribution of perceived parental acceptance on behavioral problems is organized by the three informants (mothers, fathers, and children) and by the children’s externalizing and internalizing problems.

TABLE 2

TABLE 2. Hierarchical regression analyses predicting children’s behavioral problems by multi-informants.

When the informant referencing the child’s behavioral problems is the mother, the maternal acceptance reported by mothers shows the largest increment of R², especially for externalizing problems. However, paternal acceptance reported by fathers made a significant contribution to externalizing problems (not internalizing), and maternal acceptance reported by mothers made a significant contribution to both internalizing and externalizing behavioral problems. Parental acceptance (maternal or paternal) perceived by children does not make any significant contribution to behavioral problems. Parental acceptance reported by fathers and maternal acceptance reported by mothers considered together become to explain 19% of the variance on externalizing problems.

When the informant referencing the child’s behavioral problems is the father, the same pattern was found, with the exception of the instance of externalizing problems seen in step 4, wherein children make a significant contribution. Parental acceptance reported by fathers, mothers, and children considered together become to explain the 40% of the variance on externalizing problems.

Finally, when the informant referencing the child’s behavioral problems is the child, the largest increase occurs in step 4, when children report on parental acceptance. Nevertheless, both paternal and maternal acceptances were significant predictors of externalizing problems (not internalizing problems), while only paternal acceptance was significant for internalizing problems. The increase in the variance explained by the parental acceptance perceived by children is 13% for externalizing problems and 4% for internalizing. Parental acceptance reported by fathers and children (the significant sources of information) considered together become to explain the 11% of the variance on externalizing problems and 14% on internalizing problems.

Discussion

Method effects and incremental validity are two important issues for construct validity. The analysis of empirical similarities and differences between self and others as informants contribute to the knowledge of consistency of measures, its reliability and accuracy, and its validity in terms of behavior prediction (Kenny, 1994; Neyer, 2006). This study dealt with two questions: (1) Are there significant informant effects predicting children’s behavioral problems from perceived parental acceptance? (2) What is the incremental validity of children’s perceived parental acceptance over parents’ perceived parental acceptance in predicting children’s behavioral problems?

In relation to the first question, our findings confirm a significant informant effect, which shows that the predictive values are different from one informant to the others when predicting behavioral problems in children based on perceived parental acceptance. Consequently, the magnitude of relations in terms of behavior prediction between parental acceptance and children’s externalizing and internalizing problems depends on the source of information used (i.e., children, mothers, or fathers). When the informant speaking on the child’s behavioral problems is the mother, maternal acceptance perceived by mothers and paternal acceptance perceived by fathers are the best predictors of children’s externalizing problems, while the best predictor for internalizing problems is only the maternal acceptance informed by mothers. The information provided by children about parental acceptance does not make any contribution to the behavioral problems reported upon by mothers. Likewise, the same pattern emerges when the informant about the child’s behavioral problems is the father, except that children make a significant contribution to informing on externalizing problems (not internalizing). However, when children act as informants on their own behavioral problems, the pattern found is completely different; maternal acceptance as assessed by mothers does not make any contribution to the children’s behavioral problems. Only paternal acceptance reported by fathers or children predicts the externalizing and internalizing problems; additionally, maternal acceptance reported by children predicts internalizing (not externalizing) problems.

The significant predictive value of perceived parental acceptance and children’s psychological adjustment is very well supported in family research (Khaleque and Rohner, 2012; Rohner et al., 2012), but no studies have been conducted to explore the informant effect of parental acceptance on children’s behavioral problems. Our results support this significant relation regardless of the source of information. Furthermore, our findings are consistent with previous studies that have found an informant effect reflected on the low or moderate confluence between children and parents on the information given by each of them (Achenbach et al., 1987; Rescorla et al., 2013; De Los Reyes et al., 2015). There are numerous prospective reasons for these results, such as the potential biased perception of informants (i.e., parents tending to perceive and inform about less or more problems than children), the information that informants use to rate the scales (i.e., family and school), conceptions of what constitutes abnormal behavior (Richters, 1992), the informants’ own emotional state (Chilcoat and Breslau, 1997; Najman et al., 2000; Berg-Nielsen et al., 2003), the closeness of parent–child relationships (Hughes and Gullone, 2010), or the observability of behaviors (De Los Reyes and Kazdin, 2005).

According to previous studies (Stanger et al., 1992; Duhig et al., 2000), our results support the different predictive utility that a multiaxial assessment approach may have in children’s outcomes, specifically in predicting the children’s externalizing and internalizing behavioral problems from the parental acceptance construct. In this regard, when parents report about the children’s behavioral problems, both fathers (paternal acceptance) and mothers (maternal acceptance) tend to be the best informants to predict externalizing problems, while mothers (maternal acceptance) excel at predicting internalizing ones. However, when children report about their own behavioral problems, children (paternal acceptance to externalizing and internalizing problems, and maternal acceptance to internalizing ones) and fathers (paternal acceptance) tend to be the best informants to predict all kinds of children’s behavioral problems.

Research does not yet allow us to make a conclusion about to what extent maternal or paternal acceptance will make a higher or lower contribution to children’s psychological problems. Some studies suggest that maternal parenting is more strongly associated with children’s emotional and behavioral problems than paternal parenting (Rosnati et al., 2007; Meunier et al., 2012), while other studies find that the opposite is true (Flouri and Buchanan, 2002; Khaleque and Rohner, 2011). Probably on the basis of this contribution differences could be the externalized–internalized nature of behavioral problems, as well as the informant effect. Accordingly, the greater contribution of maternal acceptance to the children’s problems could be explained by the closeness of the mother–child relationship and by the fact that mothers tend to have more knowledge about the children’s behavioral problems (mainly about the internalizing ones), possibly because mothers generally spend more time with their children than fathers (Renk et al., 2003; De Los Reyes and Kazdin, 2005), or because mothers could be perceived by their offspring to have higher interpersonal power and prestige than fathers (Carrasco et al., 2014). Paternal acceptance may become more relevant to externalizing problems than internalizing because of the nature of father–child relationships, which tend to be more focused on leisure activities (Torres et al., 2014) and goal-oriented behaviors (Leaper et al., 1998; Tenenbaum and Leaper, 2003). The informant effect that our study shows is consistent with the studies that found a higher contribution of paternal acceptance vs. maternal acceptance when the informants are children (Flouri and Buchanan, 2002; Bosco et al., 2003; Khaleque and Rohner, 2011) or teachers (Mattanah, 2001). Maternal parenting tends to be a stronger predictor of children’s behavioral problems when parents are the source of information (Gryczkowski et al., 2010), but this is not always confirmed (Hoeve et al., 2009).

Regarding the second question concerning how incremental validity was also affected by the source of information on the children’s behavioral problems, our results suggest that there are differential contributions of one source of information over the others and a subsequent incremental validity related to which combination of sources is considered. More specifically, when the informant about the child’s behavioral problems is the mother, both father’s and mother’s information about parental acceptance increases the predictive validity for externalizing problems, but only the mother’s information does this (maternal acceptance) for internalizing. However, when the informant about the child’s behavioral problems is the father, then mothers, fathers, and children increase the predictive validity for externalizing problems. Nevertheless, only the mother’s information about maternal acceptance has significant predictive value on internalizing problems. Finally, when the informant about the child’s behavioral problems is the child, then mothers, children, and fathers increase the predictive validity for externalizing problems, but only fathers (not mothers) and children do this for internalizing problems. It is important to highlight that mothers have the higher incremental validity when parents (mothers or fathers) inform about children’s problems, but that children make the larger contribution to incremental validity when they self-report about their own behavioral problems. These results support the children’s ability to be introspective and to assess their own thoughts and feelings even better than adults (Bidaut-Russell et al., 1995; Johnston and Murray, 2003). These results are also consistent with the studies that support the incremental value of adult informants compared with the child’s reports on externalizing problems (Loeber et al., 1990, 1991).

Furthermore, our results support that single informants (parents or children) produced significantly stronger effects than multiple informants (parents and children). That is, when the same informant provides information about parental acceptance (predictor) and the children’s outcomes (dependent variable), this single informant tends to reach the higher incremental validity. It is probably due to shared method variance (Campbell and Fiske, 1959). This effect may be particularly prominent when children are the source of information. Although asking children to report on parenting and their own behavioral problems can lead to inflated effect size estimates, children could provide the best information about themselves and the perceived parent–child relationships. The higher incremental validity of mothers on children’s internalizing problems is consistent with the higher predictive value of maternal acceptance on internalizing behaviors, as previously discussed.

When fathers are the source of information, the rest of the informants (children and mothers) add significant incremental validity. This could be because fathers sometimes have less knowledge of children’s day-to-day lives, meaning that more information is needed from mothers and children to predict children’s behavioral problems. However, when children are the source of information, the incremental validity is mainly added by fathers. This may be because of overlapped information from mothers and children, as these would share more information about the emotional lives of the children. It is consistent with the higher agreement between mothers and children than between fathers and children (Schneewind and Ruppert, 2013; Leung and Shek, 2014). The closer relationship of mother and child can account for a higher concurrence on the information provided by these informants, and therefore, the parent with a closer relationship will give much redundant information when added to the one given by the child. In cultures like that of Spain, where gender and parental roles are still quite differentiated, it is common for mothers to spend more time than fathers with the children, which could be a reason why the mother does not add significant information when the child is used as the primary informant. Similarly, when the mother is the primary informant, the child does not add additional significant information.

Considering all these results as a whole, it can be concluded that the child is the best source of information about parental acceptance when we are trying to predict the children’s behavioral problems (both externalizing and internalizing) reported by the own child. However, when the behavioral problems are informed by the parents, the parental acceptance information provided by them will be the data with better predictive value for children’s externalizing problems. This changes when we deal with children’s internalizing problems that are reported by the parents, in which case the mother’s information will be the most predictive one.

A few limitations should be considered for future lines of research. First, this study focused on the general population instead of a clinical sample, meaning that generalization of the current findings to clinical populations should be made with caution, and future research should consider how these two samples may differ both quantitatively and qualitatively. Second, the lack of analysis by sex and age as moderators may be particularly relevant (Crick and Grotpeter, 1995; Johnston and Murray, 2003; Hughes and Gullone, 2010) in terms of informant effect and incremental validity. Studies about sex and age differences in the perception of parental acceptance and the expression of internalizing or externalizing problems symptoms may lead to variations in informant agreement and in relationships between parental acceptance and children’s symptoms. Third, the parent’s social desirability could minimize their reports about any adverse parenting experiences (i.e., rejection) affecting the level of parent–child agreement. Four, different methods of evaluation such as observations, rating scales, and self-reports should be explored in addition to the informant method. Future studies conducted from a developmental and gender perspective with a multi-measure perspective and using clinical samples are advised in order to bring more light to the informant effect and incremental validity.

Despite the above limitations, the findings of the present study have important practical implications. Considering previous analysis, a multi-informant perspective rather than a single should be considered in order to increase the predictive value and the incremental validity when we try to predict children’s internalizing and externalizing problems. Our results suggest that mother–father or child–father informant pairs seem to be the way to optimize the combinations of sources of information in order to predict children’s behavioral problems from parental acceptance. Nevertheless, a child may give enough information to make future decisions, and if we have to add only one informant to the assessment, this should be the father. There is a clear need for more research from a multi-method perspective in the child assessment field, rather than having blind faith in a “more are better” approach to getting informants (Johnston and Murray, 2003), which will lead to an optimization of empirically based children’s assessment (Carrasco et al., 2008).

Author Contributions

The tasks of each individual author are described in the folloing lines. EI-S: Bibliographic review, preparation of data matrices, drafting the theoretical contents, drafting the discusion, writing and preparing manuscript for sending. FH-T: Collection of data, statistical analysis, drafting the methodology and results. MC: Collection of data, statistical analysis, drafting the methodology and results, theoretical contents review, team coordination.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

This research is included in the Project PSI2011-28925 and it is supported by grants from the Spanish Government, Ministerio de Ciencia e Innovación.

References

Achenbach, T. M. (2006). As others see us: clinical and research implications of cross-informant correlations for psychopathology. Curr. Dir. Psychol. Sci. 15, 94–98. doi: 10.1111/j.0963-7214.2006.00414.x

CrossRef Full Text | Google Scholar

Achenbach, T. M. (2011). Commentary: definitely more than measurement error: but how should we understand and deal with informant discrepancies? J. Clin. Child Adolesc. Psychol. 40, 80–86. doi: 10.1080/15374416.2011.533416

PubMed Abstract | CrossRef Full Text | Google Scholar

Achenbach, T. M., McConaughy, S. H., and Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol. Bull. 101, 213–232. doi: 10.1037/0033-2909.101.2.213

PubMed Abstract | CrossRef Full Text | Google Scholar

Achenbach, T. M., and Rescorla, L. A. (2007). Multicultural Supplement to the Manual for the ASEBA: School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, and Families.

Google Scholar

Berg-Nielsen, T. S., Vika, A., and Dahl, A. (2003). When adolescents disagree with their mothers: CBCL-YSR discrepancies related to maternal depression and adolescent self-esteem. Child Care Health Dev. 29, 207–213. doi: 10.1046/j.1365-2214.2003.00332.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Bidaut-Russell, M., Reich, W., Cottler, L. B., Robins, L. N., Compton, W. M., and Mattison, R. E. (1995). The Diagnostic Interview Schedule for Children (PC-DISC v. 3.0): parents and adolescents suggest reasons for expecting discrepant answers. J. Abnorm. Child Psychol. 23, 641–659. doi: 10.1007/BF01447667

PubMed Abstract | CrossRef Full Text | Google Scholar

Biederman, J., Keenan, K., and Faraone, S. V. (1990). Parent-based diagnosis of attention deficit disorder predicts a diagnosis based on teacher report. J. Am. Acad. Child Adolesc. Psychiatry 29, 698–701. doi: 10.1097/00004583-199009000-00004

PubMed Abstract | CrossRef Full Text | Google Scholar

Bosco, G. L., Renk, K., Dinger, T. M., Epstein, M. K., and Phares, V. (2003). The connections between adolescents’ perceptions of parents, parental psychological symptoms, and adolescent functioning. J. Appl. Dev. Psychol. 24, 179–200. doi: 10.1016/S0193-3973(03)00044-3

CrossRef Full Text | Google Scholar

Byrne, B. (1998). Structural Equation Modelling with LISREL, PRELIS y SIMPLIS: Basic Concepts, Applications and Programming. London: LEA.

Google Scholar

Campbell, D. T., and Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol. Bull. 56, 81–105. doi: 10.1037/h0046016

CrossRef Full Text | Google Scholar

Cantwell, D. P., Lewinsohn, P. M., Rohde, P., and Seeley, J. R. (1997). Correspondence between adolescent report and parent report of psychiatric diagnostic data. J. Am. Acad. Child Adolesc. Psychiatry 36, 610–619. doi: 10.1097/00004583-199705000-00011

PubMed Abstract | CrossRef Full Text | Google Scholar

Carrasco, M. A., Holgado, F. P., Del Barrio, M. V., and Barbero, M. I (2008). Validez incremental: un estudio aplicado con diversas fuentes informantes y medidas [Incremental validity: an applied study using different informants and measurements]. Acción Psicol. 5, 65–76.

Google Scholar

Carrasco, M. Á., Holgado, F. P., and Delgado, B. (2014). Cuestionario interpersonal de poder y prestigio parental (3PQ): dimensionalidad y propiedades psicométricas en niños españoles [The perceived interpersonal parental power and prestige questionnaire (3PQ): dimensionality and psychometric properties in Spain]. Acción Psicol. 11, 47–60. doi: 10.5944/ap.11.2.14174

CrossRef Full Text | Google Scholar

Cheng, H., and Furnham, A. (2004). Perceived parental rearing style, self-esteem and self-criticism as predictors of happiness. J. Happiness Stud. 5, 1–21. doi: 10.1023/B:JOHS.0000021704.35267.05

CrossRef Full Text | Google Scholar

Chilcoat, H. D., and Breslau, N. (1997). Does psychiatric history bias mothers’ reports? An application of a new analytic approach. J. Am. Acad. Child Adolesc. Psychiatry 36, 971–979.

Google Scholar

Crick, N. R., and Grotpeter, J. K. (1995). Relational aggression, gender, and social-psychological adjustment. Child Dev. 66, 710–722. doi: 10.2307/1131945

CrossRef Full Text | Google Scholar

De Los Reyes, A. (2011). Introduction to the special section: more than measurement error: discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. J. Clin. Child Adolesc. Psychol. 40, 1–9. doi: 10.1080/15374416.2011.533405

PubMed Abstract | CrossRef Full Text | Google Scholar

De Los Reyes, A., Augenstein, T. M., Wang, M., Thomas, S. A., Drabick, D. A. G., Burgers, D. E., et al. (2015). The validity of the multi-informant approach to assessing child and adolescent mental health. Psychol. Bull. 144, 858–900. doi: 10.1037/a0038498

PubMed Abstract | CrossRef Full Text | Google Scholar

De Los Reyes, A., and Kazdin, A. E. (2004). Measuring informant discrepancies in clinical child research. Psychol. Assess. 16, 330–334. doi: 10.1037/1040-3590.16.3.330

PubMed Abstract | CrossRef Full Text | Google Scholar

De Los Reyes, A., and Kazdin, A. E. (2005). Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychol. Bull. 131, 483–509. doi: 10.1037/0033-2909.131.4.483

PubMed Abstract | CrossRef Full Text | Google Scholar

De Los Reyes, A., and Kazdin, A. E. (2006). Informant discrepancies in assessing child dysfunction relate to dysfunction within mother-child interactions. J. Child Fam. Stud. 15, 645–663. doi: 10.1007/s10826-006-9031-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Del Barrio, V. D., Ramírez-Uclés, I., Romero, C., and Carrasco, M. A. (2014). Adaptación del child-PARQ/Control: versiones para el padre y la madre en población infantil y adolescente española [Adaptation of the child-PARQ/Control mother and father versions in Spanish child and adolescent population]. Acción Psicol. 11, 27–46. doi: 10.5944/ap.11.2.14173

PubMed Abstract | CrossRef Full Text | Google Scholar

Duhig, A. M., Renk, K., Epstein, M. K., and Phares, V. (2000). Interparental agreement on internalizing, externalizing, and total behavior problems: a meta-analysis. Clin. Psychol. Sci. Pract. 7, 435–453. doi: 10.1093/clipsy.7.4.435

CrossRef Full Text | Google Scholar

Eid, M. E., and Diener, E. E. (eds) (2006). Handbook of Multimethod Measurement in Psychology. Washington, DC: American Psychological Association.

Google Scholar

Epkins, C. (1993). A preliminary comparison of teacher ratings and child self-report of depression, anxiety, and aggression in inpatient and elementary school samples. J. Abnorm. Child Psychol. 21, 649–661. doi: 10.1007/BF00916448

PubMed Abstract | CrossRef Full Text | Google Scholar

Eyde, L. D., Robertson, G. J., Krug, S. E., Moreland, K. L., Robertson, A. G., Shewan, C. M., et al. (1993). Responsible Test Use: Case Studies for Assessing Human Behavior. Washington, DC: American Psychological Association.

Google Scholar

Flouri, E., and Buchanan, A. (2002). What predicts good relationships with parents in adolescents and partners in adult life: findings from the 1958 British Birth Cohort. J. Fam. Psychol. 16, 196–198. doi: 10.1037/0893-3200.16.2.186

PubMed Abstract | CrossRef Full Text | Google Scholar

Garb, H. N. (2003). Incremental validity and the assessment of psychopathology in adults. Psychol. Assess. 15, 508–520. doi: 10.1037/1040-3590.15.4.508

PubMed Abstract | CrossRef Full Text | Google Scholar

Goodman, K. L., De Los Reyes, A., and Bradshaw, C. P. (2010). Understanding and using informants’ reporting discrepancies of youth victimization: a conceptual model and recommendations for research. Clin. Child Fam. Psychol. Rev. 13, 366–383. doi: 10.1007/s10567-010-0076-x

PubMed Abstract | CrossRef Full Text | Google Scholar

Grigorenko, E. L., Geiser, C., Slobodskaya, H. R., and Francis, D. J. (2010). Cross-informant symptoms from CBCL, TRF, and YSR: trait and method variance in a normative sample of Russian youths. Psychol. Assess. 22, 893–911. doi: 10.1037/a0020703

PubMed Abstract | CrossRef Full Text | Google Scholar

Gryczkowski, M. R., Jordan, S. S., and Mercer, S. H. (2010). Differential relations between mothers’ and fathers’ parenting practices and child externalizing behavior. J. Child Fam. Stud. 19, 539–546. doi: 10.1007/s10826-009-9326-2

CrossRef Full Text | Google Scholar

Hoeve, M., Dubas, J. S., Eichelsheim, V. I., Laan, P. H. V. D., Smeenk, W., and Gerris, J. R. M. (2009). The relationship between parenting and delinquency: a meta-analysis. J. Abnorm. Child Psychol. 37, 749–775. doi: 10.1007/s10802-009-9310-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Hughes, E. K., and Gullone, E. (2010). Discrepancies between adolescent, mother, and father reports of adolescent internalizing symptom levels and their association with parent symptoms. J. Clin. Psychol. 66, 978–995. doi: 10.1002/jclp.20695

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunsley, J. (2002). Psychological testing and psychological assessment: a closer examination. Am. Psychol. 57, 139–140. doi: 10.1037/0003-066X.57.2.139

CrossRef Full Text | Google Scholar

Hunsley, J. (2003). Introduction to the special section on incremental validity and utility in clinical assessment. Psychol. Assess. 15, 443–445. doi: 10.1037/1040-3590.15.4.443

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunsley, J., and Mash, E. J. (2005). Introduction to the special section on developing guidelines for the evidence based assessment (EBA) of adult disorders. Psychol. Assess. 17, 251–255. doi: 10.1037/1040-3590.17.3.251

PubMed Abstract | CrossRef Full Text | Google Scholar

Hunsley, J., and Mash, E. J. (2007). Evidence-based assessment. Annu. Rev. Clin. Psychol. 3, 29–51. doi: 10.1146/annurev.clinpsy.3.022806.091419

CrossRef Full Text | Google Scholar

Jané, M., Araneda, N., Valero, S., and Doménech, E. (2000). Evaluación de la sintomatología depresiva del preescolar: correspondencia entre los informes de padres y de maestros. [Assessment of preschool symptomatology depression: correspondence between reports by parents and teachers]. Psicothema 12, 212–215.

Google Scholar

Johnston, C. H., and Murray, C. (2003). Incremental validity in the psychological assessment of children and adolescents. Psychol. Assess. 15, 496–507. doi: 10.1037/1040-3590.15.4.496

PubMed Abstract | CrossRef Full Text | Google Scholar

Kenny, D. A. (1994). Interpersonal Perception: A Social Relations Analysis. New York, NY: Guilford Press.

Google Scholar

Khaleque, A. (2015). Perceived parental neglect, and children’s psychological maladjustment, and negative personality dispositions: a meta-analysis of multi-cultural studies. J. Child Fam. Stud. 24, 1419–1428. doi: 10.1007/s10826-014-9948-x

CrossRef Full Text | Google Scholar

Khaleque, A., and Rohner, R. P. (2002). Perceived parental acceptance rejection and psychological adjustment: a meta-analysis of cross-cultural and intracultural studies. J. Marriage Fam. 64, 54–64. doi: 10.1111/j.1741-3737.2002.00054.x

CrossRef Full Text | Google Scholar

Khaleque, A., and Rohner, R. P. (2011). Transnational relations between perceived parental acceptance and personality dispositions of children and adults: a meta-analytic review. Pers. Soc. Psychol. Rev. 16, 103–115. doi: 10.1177/1088868311418986

PubMed Abstract | CrossRef Full Text | Google Scholar

Khaleque, A., and Rohner, R. P. (2012). Pancultural associations between perceived parental acceptance and psychological adjustment of children and adults: a meta-analytic review of worldwide research. J. Cross Cult. Psychol. 43, 784–800. doi: 10.1177/0022022111406120

CrossRef Full Text | Google Scholar

Klein, R. G. (1991). Parent-child agreement in clinical assessment of anxiety and other psychopathology: a review. J. Anxiety Disord. 5, 187–198. doi: 10.1016/0887-6185(91)90028-R

CrossRef Full Text | Google Scholar

Langhinrichsen, J., Lichtenstein, E., Seely, J. R., Hops, H., Ary, D. V., Tildesley, E., et al. (1990). Parent-adolescent congruence for adolescent substance abuse. J. Youth Adolesc. 19, 623–635. doi: 10.1007/BF01537181

PubMed Abstract | CrossRef Full Text | Google Scholar

Lanz, M., Scabini, E., Vermulst, A. A., and Gerris, J. R. M. (2001). Congruence on child rearing in families with early adolescent and middle adolescent children. Int. J. Behav. Dev. 25, 133–139. doi: 10.1080/01650250042000104

CrossRef Full Text | Google Scholar

Lapouse, R., and Monk, M. A. (1958). An epidemiologic study of behavior characteristics in children. Am. J. Public Health Nations Health 48, 1134–1144. doi: 10.2105/AJPH.48.9.1134

CrossRef Full Text | Google Scholar

Leaper, C., Anderson, K. J., and Sanders, P. (1998). Moderators of gender effects on parents’ talk to their children. Dev. Psychol. 34, 3–27. doi: 10.1037/0012-1649.34.1.3

CrossRef Full Text | Google Scholar

Leung, J. T., and Shek, D. T. (2014). Parent-adolescent discrepancies in perceived parenting characteristics and adolescent developmental outcomes in poor chinese families. J. Child Fam. Stud. 23, 200–213. doi: 10.1007/s10826-013-9775-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Likert, R. S. (1932). Technique for the measurement of attitudes. Arch. Psychol. 140, 44–53.

Google Scholar

Loeber, R., Green, S. M., and Lahey, B. B. (1990). Mental health professionals’ perception of the utility of children, mothers, and teachers as informants on childhood psychopathology. J. Clin. Child Psychol. 19, 136–143.

Google Scholar

Loeber, R., Green, S. M., Lahey, B. B., and Stouthamer-Loeber, M. (1991). Differences and similarities between children, mothers and teachers as informants on disruptive behaviour disorders. J. Abnorm. Child Psychol. 19, 75–95. doi: 10.1007/BF00910566

PubMed Abstract | CrossRef Full Text | Google Scholar

Markon, K. E., Chmielewski, M., and Miller, C. J. (2011). The reliability and validity of discrete and continuous measures of psychopathology: a quantitative review. Psychol. Bull. 137, 856–879. doi: 10.1037/a0023678

PubMed Abstract | CrossRef Full Text | Google Scholar

Mash, E. J., and Terdal, L. G. (1997). Assessment of Childhood Disorders, 3rd Edn. New York, NY: Guilford Press.

Google Scholar

Mattanah, J. F. (2001). Parental psychological autonomy and children’s academic competence and behavioral adjustment in late childhood: more than just limit-setting and warmth. Merrill Parker Q. 47, 355–376. doi: 10.1353/mpq.2001.0017

CrossRef Full Text | Google Scholar

McLeod, B. D., Weisz, J. R., and Wood, J. J. (2007). Examining the association between parenting and childhood depression: a meta-analysis. Clin. Psychol. Rev. 27, 986–1003. doi: 10.1016/j.cpr.2006.09.002

PubMed Abstract | CrossRef Full Text | Google Scholar

Meunier, J. C. H., Bisceglia, R., and Jenkis, J. M. (2012). Differential parenting and children’s behavioral problems: curvilinear associations and mother–father combined effects. Dev. Psychol. 48, 987–1002. doi: 10.1037/a0026321

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, G. J., and Archer, R. P. (2001). The hard science of Rorschach research: what do we know and where do we go? Psychol. Assess. 13, 486–502. doi: 10.1037/1040-3590.13.4.486

PubMed Abstract | CrossRef Full Text | Google Scholar

Meyer, G. J., Finn, S. E., Eyde, L. D., Kay, G. G., Moreland, K. L., Dies, R. R., et al. (2001). Psychological testing and psychological assessment: a review of evidence and issues. Am. Psychol. 56, 128–165. doi: 10.1037/0003-066X.56.2.128

CrossRef Full Text | Google Scholar

Najman, J. M., Williams, G. M., Nikles, J., Spence, S., Bor, W., O’Callaghan, M., et al. (2000). Mothers’ mental illness and child behavior problems: cause-effect association or observation bias? J. Am. Acad. Child Adolesc. Psychiatry 39, 592–602. doi: 10.1097/00004583-200005000-00013

PubMed Abstract | CrossRef Full Text | Google Scholar

Neyer, F. J. (2006). “Informant assessment,” in Handbook of Multimethod Measurement in Psychology, eds M. E. Eid and E. E. Diener (Washington, DC: American Psychological Association), 43–59.

Google Scholar

Ollendick, T. H., and Hersen, M. E. (1993). Handbook of Child and Adolescent Assessment. Boston, MA: Allyn & Bacon.

Google Scholar

Ramírez-Lucas, A., Ferrando, M., and Sainz, A. (2015). Influyen los estilos parentales y la inteligencia emocional de los padres en el desarrollo emocional de sus hijos escolarizados en 2° ciclo de educación infantil? [Do parental styles and parents’ emotional intelligence influence their children’s emotional development in kindergarten school?]. Acción Psicol. 12, 65–78. doi: 10.5944/ap.12.1.14314

CrossRef Full Text | Google Scholar

Renk, K., Roberts, R., Roddenberry, A., Luick, M., Hillhouse, S., Meehan, C., et al. (2003). Mothers, fathers, gender role, and time parents spend with their children. Sex Roles 48, 305–315. doi: 10.1023/A:1022934412910

CrossRef Full Text | Google Scholar

Rescorla, L. A., Ginzburg, S., Achenbach, T. M., Ivanova, M. Y., Almqvist, F., Begovac, I., et al. (2013). Cross-informant agreement between parent-reported and adolescent self-reported problems in 25 societies. J. Clin. Child Adolesc. Psychol. 42, 262–273. doi: 10.1080/15374416.2012.717870

PubMed Abstract | CrossRef Full Text | Google Scholar

Richters, J. E. (1992). Depressed mothers as informants about their children: a critical review of the evidence for distortion. Psychol. Bull. 112, 485–499. doi: 10.1037/0033-2909.112.3.485

PubMed Abstract | CrossRef Full Text | Google Scholar

Rohner, R. P. (1986). The Warmth Dimension: Foundations of Parental Acceptance-Rejection Theory. Beverly Hills, CA: Sage Publications, Inc.

Google Scholar

Rohner, R. P. (1990). Handbook for the Study of Parental Acceptance and Rejection. Storrs, CT: University of Connecticut.

Google Scholar

Rohner, R. P., and Khaleque, A. (eds) (2005). Handbook for the Study of Parental Acceptance and Rejection, 4th Edn. Storrs, CT: Rohner Research Publications, 187–226.

Google Scholar

Rohner, R. P., and Khaleque, A. (2010). Testing central postulates of parental acceptance-rejection theory (PARTheory): a meta-analysis of cross-cultural studies. J. Fam. Theory Rev. 3, 73–87. doi: 10.1111/j.1756-2589.2010.00040.x

CrossRef Full Text | Google Scholar

Rohner, R. P., Khaleque, A., and Cournoyer, D. E. (2012). “Parental acceptance- rejection theory, methods, and implications,” in Handbook for the Study of Parental Acceptance and Rejection, 4th Edn, eds R. P. Rohner and A. Khaleque (Storrs, CT: Rohner Research Publications), 1–35.

Google Scholar

Rosnati, R., Iafrate, R., and Scabini, E. (2007). Parent–adolescent communication in foster, inter-country adoptive, and biological Italian families: gender and generational differences. Int. J. Psychol. 42, 36–45. doi: 10.1080/00207590500412128

PubMed Abstract | CrossRef Full Text | Google Scholar

Schneewind, K. A., and Ruppert, S. (2013). Personality and Family Development: An Intergenerational Longitudinal Comparison. New York, NY: Psychology Press.

Google Scholar

Stanger, C., and Lewis, M. (1993). Agreement among parents, teachers, and children on internalizing and externalizing behavior problems. J. Clin. Child Psychol. 22, 107–116. doi: 10.1207/s15374424jccp2201_11

CrossRef Full Text | Google Scholar

Stanger, C., McConaughy, S. H., and Achenbach, T. M. (1992). Three-year course of behavioral/emotional problems in a national sample of 4- to 16-year-olds: II. Predictors of syndromes. J. Am. Acad. Child Adolesc. Psychiatry 31, 941–950. doi: 10.1097/00004583-199209000-00024

PubMed Abstract | CrossRef Full Text | Google Scholar

Tarver, J., Daley, D., Lockwood, J., and Sayal, K. (2014). Are self-directed parenting interventions sufficient for externalising behaviour problems in childhood? A systematic review and meta-analysis. Eur. Child Adolesc. Psychiatry 23, 1123–1137. doi: 10.1007/s00787-014-0556-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Tenenbaum, H. R., and Leaper, C. (2003). Parent-child conversations about science: the socialization of gender inequities? Dev. Psychol. 39, 34–47. doi: 10.1037/0012-1649.39.1.34

PubMed Abstract | CrossRef Full Text | Google Scholar

Torres, N., Veríssimo, M., Monteiro, L., Ribeiro, O., and Santos, A. J. (2014). Domains of father involvement, social competence and problem behavior in preschool children. J. Fam. Stud. 20, 188–203. doi: 10.5172/jfs.2014.20.3.188

CrossRef Full Text | Google Scholar

Weitkamp, K., Daniels, J., Rosenthal, S., Romer, G., and Wiegand-Grefe, S. (2013). Health-related quality of life: cross-informant agreement of father, mother, and self-report for children and adolescents in outpatient psychotherapy treatment. Child Adolesc. Ment. Health 18, 88–94. doi: 10.1111/j.1475-3588.2012.00656.x

CrossRef Full Text | Google Scholar