Skip to main content


Front. Psychiatry, 24 September 2019
Sec. Child and Adolescent Psychiatry
This article is part of the Research Topic Bullying by Peers and in the Family: Precursors, Consequences and Intervention View all 13 articles

Frequent Bullying Involvement and Brain Morphology in Children

Ryan L. Muetzel,Ryan L. Muetzel1,2Rosa H. Mulder,,Rosa H. Mulder1,3,4Sander Lamballais,Sander Lamballais2,4Andrea P. Cortes Hidalgo,Andrea P. Cortes Hidalgo1,4Pauline Jansen,Pauline Jansen1,5Berna GüroluBerna Güroğlu6Meike W. Vernooiji,Meike W. Vernooiji2,7Manon Hillegers,Manon Hillegers1,4Tonya White,Tonya White1,7Hanan El Marroun,,Hanan El Marroun1,5,8Henning Tiemeier,*Henning Tiemeier1,9*
  • 1Department of Child and Adolescent Psychiatry/Psychology, Erasmus MC University Medical Center, Rotterdam, Netherlands
  • 2Department of Epidemiology, Erasmus MC University Medical Center, Rotterdam, Netherlands
  • 3Institute of Education and Child Studies, Leiden University, Leiden, Netherlands
  • 4Generation R Study Group, Erasmus MC University Medical Center, Rotterdam, Netherlands
  • 5Department of Psychology, Education and Child Studies, Erasmus School of Social and Behavioral Sciences, Erasmus University Rotterdam, Rotterdam, Netherlands
  • 6Institute of Psychology, Leiden University, Leiden, Netherlands
  • 7Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Center, Rotterdam, Netherlands
  • 8Department of Pediatrics, Erasmus MC University Medical Center, Rotterdam, Netherlands
  • 9The Department of Social and Behavioral Science, Harvard TH Chan School of Public Health, Boston, MA, United States

Background: Over the past few decades, bullying has been recognized as a considerable public health concern. Involvement in bullying is associated with poor long-term social and psychiatric outcomes for both perpetrators and targets of bullying. Despite this concerning prognosis, few studies have investigated possible neurobiological correlates of bullying involvement that may explain the long-term impact of bullying. Cortical thickness is ideally suited for examining deviations in typical brain development, as it has been shown to detect subtle differences in children with psychopathology. We tested associations between bullying involvement and cortical thickness using a large, population-based cohort.

Methods: The study sample consisted of 2,602 participants from the Generation R Study. When children were 8 years old, parents and teachers reported on common forms of child bullying involvement (physical, verbal, and relational). Questions ascertained whether a child was involved as a perpetrator (n = 82), a target of bullying (n = 92), as a combined perpetrator and target of bullying (n = 47), or uninvolved in frequent bullying (n = 2,381). High-resolution structural MRI was conducted when children were 10 years of age. Cortical thickness estimates across the cortical mantle were compared among groups.

Results: Children classified as frequent targets of bullying showed thicker cortex in the fusiform gyrus compared to those uninvolved in bullying (B = 0.108, pcorrected < 0.001). Results remained consistent when adjusted for socioeconomic factors, general intelligence, and psychiatric symptoms. Children classified as frequent perpetrators showed thinner cortex in the cuneus region; however, this association did not survive stringent correction for multiple testing. Lastly, no differences in cortical thickness were observed in perpetrator–targets.

Discussion: Bullying involvement in young children was associated with differential cortical morphology. Specifically, the fusiform gyrus, often involved in facial processing, showed thicker cortex in targets of frequent bullying. Longitudinal data are necessary to demonstrate the temporality of the underlying neurobiology associated with bullying involvement.


The past decades have witnessed bullying during childhood emerge as a considerable public health concern. Prevalence estimates are relatively high, although vary considerably by age, gender, frequency of involvement, and country (1). In addition to the immediate burden on the child, poor long-term outcomes have been consistently reported in those involved once they reach adulthood, including increased rates of psychiatric disorders, substance abuse, problems with social functioning, and suicidality (27). The persistence of these problems into adulthood suggests that a potential underlying neurobiological substrate may be linked to bullying involvement.

Bullying in children is formally characterized as unwanted, repeated, and aggressive behavior among peers which occurs in the context of an actual or perceived power imbalance (8). Involvement takes place in multiple forms, including physical (e.g., hitting, fighting), verbal (e.g., name calling, inappropriate comments), relational (e.g., social exclusion), and more. Those involved in bullying are often classified as being a bully (perpetrator), a victim (target of bullying), or involved in both forms as a perpetrator–target (9). Against the background of high prevalence estimates and the advent of cyber bullying, it is crucial to better understand bullying in the context of neural correlates, as such features could eventually help to predict and even explain the persistent psychosocial outcomes of bullying involvement (10).

In vivo structural brain imaging methods have proven effective in examining typical (11) and atypical morphological brain development (12) and are a promising tool for ascertaining any neural correlates of bullying involvement. Previous work has already shown how early-life adversities, such as abuse, early life stress, quality of maternal care, and growing up in institutional care, impact cortical and subcortical development in children (1315). Despite this work demonstrating the sensitivity of structural neuroimaging to detect subtle morphological features of typical and atypical brain development, few studies have explored to what extent bullying involvement is associated with brain morphology and brain structure (16, 17). More substantial focus has been given to aspects of peer and social interaction using functional MRI (18) where, for example, anterior cingulate and prefrontal cortices have been implicated with differential functional activity in the context of exposure to social exclusion (19, 20). Recently, a large study of adolescents examined how structural brain volumes were related to peer victimization and psychopathology; changes in brain volumes which were related to peer victimization (specifically portions of the basal ganglia) were also predictive of internalizing problems later in life (21).

We aimed to study the association between bullying involvement and brain morphology in a large population-based setting. Parent- and teacher-rated bullying involvement was used to classify children as perpetrators, targets of bullying, or combined perpetrator–targets. We performed structural MRI to quantitatively assess the thickness of the cortical mantle, as well as hippocampal and amygdala volume; metrics shown to be associated with psychopathology and symptomatology in children. We hypothesized that targets of bullying involvement would display differences in cortical thickness in brain areas related to threat perception and sensitivity, fear, anxiety, emotional face processing, and emotional regulation (e.g., prefrontal cortex, cingulate gyrus, fusiform face area, and insula). We also hypothesized that perpetrators would differ in cortical thickness in areas related to emotional (dys)regulation (e.g., prefrontal cortex). Lastly, we hypothesized that those involved as perpetrator–targets would display the largest differences in cortical thickness in regions observed in both perpetrators and targets of bullying.

Methods and Materials


Participants in this study were part of the Generation R Study, a prospective prenatal birth cohort in Rotterdam, The Netherlands (22). When children were between the ages of 7 and 8, parents and teachers completed a questionnaire on children’s bullying involvement. At the age of 10, children visited our research-dedicated facility for a detailed behavioral assessment (23) and also underwent MRI (24). Of the 3,992 children who visited our MRI facility, 807 datasets were excluded due missing complete T1 scan (n = 114; 3%), a different T1 acquisition (n = 22, 0.6%), poor/insufficient data quality (n = 644; 16%), or incidental findings (n = 27; 0.7%, Supplemental Table S1). Of the remaining 3,185 children who had MRI data, 2,602 also had parent or teacher report information on bullying involvement and comprised the final study population. The flow chart depicted in Figure 1 illustrates these exclusions in detail. The Medical Ethics Committee of the Erasmus Medical Center approved all study procedures, and all parents and children provided written informed consent and assent, respectively.


Figure 1 Flow chart indicating participant inclusion/exclusion from study population.

Bullying Involvement Assessment

Three common forms of bullying involvement (9) were assessed using two separate informants: Physical, verbal, and relational bullying involvement were assessed by asking the child’s primary caregiver (most often the mother) and/or the child’s teacher. Separate questions ascertained whether the child was involved as a perpetrator or a target of bullying for each type of involvement (physical, verbal, relational involvement, for a total of six items). For example, parents and teachers were asked, “In the past few months, how often has your child been bullied by insults, being called names or being laughed at?” Full questions are presented in the Supplemental Material. Teachers were additionally asked whether a child was involved in material bullying. However, this item was not administered in the parent version of the questionnaire given the low endorsement rates by teachers and, therefore, was not used in the current analyses. Each item was rated on a 4-point scale ranging from “Never or less than once per month” to “More than twice per week” by the teachers and on a 5-point scale ranging from “Never” to “Several times per week” by the parents. Children were classified as perpetrators if their parent or their teacher indicated they physically, verbally, or relationally bullied other children once per week or more, which represents frequent involvement. In the event of disagreement between informants, children were classified as perpetrators if one informant indicated involvement (3, 25). Similarly, children were classified as targets of bullying if their parent or teacher indicated they were bullied by another child once per week or more. If a child was classified as both a perpetrator and as a target of bullying based on these criteria, they were reclassified as being involved as both, referred to as a perpetrator–target. Ratings from multiple informants were not available for all children, with roughly 33% receiving information from both informants, 53% receiving information from mothers only, and 14% receiving information from the teacher only. Cohen’s Kappa was in line with previous work, κ = 0.11, p < 0.05 (2528). Although this agreement is relatively low, as highlighted by previous work (25), it is also consistent with agreement between informants in the context of behavioral and emotional problems (29).

Image Acquisition

Neuroimaging data were collected on a study-dedicated, 3-Tesla MRI system (MR-750W, General Electric, Milwaukee, WI, US) using an eight-channel, receive-only head coil (24). Before scanning, children underwent a mock scanning session in order to familiarize them with the procedure and scanning environment. High-resolution, T1-weigthed structural MRI data were acquired using a coronal inversion recovery fast spoiled gradient recalled sequence with the following parameters: GE option BRAVO, TR = 8.77 ms, TE = 3.4 ms, TI = 600 ms, flip angle = 10°, matrix size = 220 × 220, field of view = 220 mm × 220 mm, slice thickness = 1 mm, number of slices = 230, ARC acceleration factor = 2.

Image Processing

Images were processed using the FreeSurfer version 6.0 analysis suite (30). First, DICOM data were converted to “MGZ” file format using the FreeSurfer “mri_convert” tool. The standard reconstruction was then conducted, where nonbrain tissue was removed, voxel intensities were corrected for B1 field inhomogeneities, voxels were segmented into white matter, gray matter, and cerebral spinal fluid, and surface-based models of white matter and gray matter were generated. Subcortical structures were automatically labeled, and volumes in cubic millimeter were extracted for the hippocampus and amygdala for this study. Cortical thickness was estimated at each point (vertex) along the cortical ribbon, and each point was also automatically assigned an anatomical label according to a predefined atlas (31). Thickness data for each participant were coregistered to a standard stereotaxic space and smoothed with a 10-mm full-width half-maximum Gaussian kernel. Cortical surface reconstructions were visually inspected for inaccuracies (32), and 16% of the scans were labeled as inadequate for data analyses.


Date of birth and sex were determined from medical records obtained at birth, and child ethnicity was defined based on the birth country of the parents. Maternal education level, a proxy for socioeconomic status, was assessed by questionnaire. Child nonverbal IQ was estimated using subtests from the Snijders–Oomen nonverbal intelligence test at the age-6 assessment (33). Lastly, child psychiatric symptoms were assessed using the parent-report Child Behavior Checklist (CBCL) administered at the age-10 assessment. The CBCL is a 100-item parental report of child behavioral and emotional problems which uses a Likert response format. The CBCL assesses a variety of domains, including internalizing (e.g., depressive/anxiety symptoms) and externalizing (e.g., attention problems). The square root transformed sum of all items (Total Problem Score) was utilized (34). For supplemental analyses (see statistical section below), additional covariates were tested for their impact on model estimates. First, for analyses involving targets of bullying, the CBCL Broadband Internalizing scale was used, as it focuses on emotional problems more common in this group. With the same rationale, the CBCL Broadband Externalizing scale was added in analyses of perpetrators of bullying. In order to rule out that childhood trauma explained any observed associations, exposure to physical and sexual abuse, derived from a retrospective parental-report of life events, was included. Briefly, a dichotomous (exposed/unexposed) variable was created if any of four items related to physical and sexual abuse were endorsed (35). Lastly, body mass index, estimated from height and weight measured at the age-6 assessment and normalized in accordance to Dutch growth curves for age and sex, was also included as a covariate in supplemental analyses.

Statistical Analysis

Analyses were run using the R statistical software [version 3.4.3 (36)]. Multiple linear regression was used for analyses of hippocampal and amygdala volume. A custom in-house package was developed to run multiple linear regression at each cortical vertex (“QDECR,” A dichotomous variable for each of the three groups (perpetrator, target of bullying, and perpetrator–target) was created and reference coded to the individuals uninvolved in bullying. Regression analyses were run in three steps to adjust for potential confounding factors. All three models are presented in order to show the impact different confounding factors have on regression coefficients, with large changes in estimates indicative of the potential for residual confounding (remaining bias in estimates). Furthermore, as different neuroimaging studies often have limited data on various confounding factors, presenting analyses in this way allows for maximal comparison with existing/new literature, for example in the frequent case when only age and sex are available. Primary analyses were adjusted for age at MRI scan, sex, and child ethnicity (model 1). Additional analyses were run by further adjusting model 1 for maternal education and child nonverbal IQ (model 2). Lastly, to determine whether results were explained by child behavior problems, model 2 was additionally adjusted for child behavior problems (model 3).

In order to ascertain to what extent the perpetrator–target classification influenced results, sensitivity analyses were run where the perpetrator–target category was not considered. These children were dichotomously classified as targets of bullying or perpetrators. In further sensitivity analyses, continuous sum scores of bullying involvement were entered into regression models in order to complement categorical analyses. In order to determine whether observed associations were different between boys and girls, models were also run with a perpetrator group-by-sex interaction term. As internalizing problems are more often related to victimization and externalizing problems are more often related to perpetrator behavior, sensitivity analyses were run for model 3 where the CBCL total problems score was replaced with either the broadband internalizing score (targets of bullying) or the broadband externalizing score (perpetrators). Lastly, in order to rule out other potential confounding factors, exposure to traumatic events as well as body mass index were added to model 3 to ensure these factors did not account for any observed associations.

Given the large number of statistical tests, analyses were adjusted for multiple comparisons using Gaussian Monte Carlo simulations (37). Clusterwise p values were Bonferroni corrected for two hemispheres (p < 0.025), and, as it has shown high correspondence with actual permutation testing at the smoothing kernel used, a cluster-forming threshold (CFT) of p = 0.001 was selected for significance testing (38). As this threshold may be conservative, in line with genome-wide association studies, a “suggestive” yet still strict CFT was also employed (p = 0.005 CFT). For illustrative purposes, different CFTs are also displayed in figures and tables when a cluster was significant at the suggestive CFT 0.005 or below.

Missing Data

Data were missing on covariates in a subset of participants for ethnicity, maternal education, nonverbal IQ and behavioral problems. In all cases, missingness was <11%. In order to retain the largest possible sample for linear regression analyses, these missing data were imputed utilizing the “mice” (multiple imputation by chained equations) package for multiple imputation (39). A number of variables that are correlated with these covariates were used in the imputation process. With 100 iterations, a total of 30 imputed datasets were generated, and results were pooled at each vertex using established methods (40).


Nonresponse was described with two sets of analyses: first, a comparison with children who participated in the age-6 assessment (roughly the age when the bullying assessment was conducted) but do not have MRI data at age 10 and, second, a comparison with children who participated in the MRI study but were excluded from analyses (e.g., due to poor data quality). Children who participated in the age-6 assessment but not in the current study had lower IQ (MMRI = 104, Mnonresponder = 99, p < 0.05), higher total behavioral problem scores (MMRI = 16.7, Mnonresponder = 18.5, p < 0.05), were less likely to be Dutch (PMRI = 64%, Pnonresponder = 53%, p < 0.05), and their mothers were less likely to have acquired higher education (PMRI = 63%, Pnonresponder = 52%, p < 0.05). Similarly, children who were excluded from the present study (e.g., because of motion artifact or missing a bullying assessment) tended to also have lower IQ (MMRI = 104, Mexcluded = 100, p < 0.05), higher total CBCL problem scores (MMRI = 16.7, Mexcluded = 18.3, p < 0.05), less likely to be Dutch (PMRI = 64%, Pexcluded = 52%, p < 0.05), and their mothers were less likely to have acquired higher education (PMRI = 63%, Pexcluded = 54%, p < 0.05).


Girls represented 51% of the sample, and children were on average 10.1 years (range, 8.5–11.9) old at the MRI visit. Based on parent report (mean age of child, 8.1 years; range, 7.5–9.9 years) and/or teacher report (mean age of child, 6.6 years; range, 4.6–9.6 years), 92 children (3.5%) were frequently involved as targets of bullying, 82 as perpetrators (3.2%), 47 as perpetrator–targets (1.8%), and 2,382 (91.5%) were uninvolved in frequent bullying. Table 1 provides a detailed overview of the sample characteristics.


Table 1 Sample Characteristics.

Targets of Bullying

Whole-brain vertex-wise analyses of cortical thickness showed that children identified as targets of bullying had thicker cortex in the left fusiform gyrus compared to those uninvolved in frequent bullying (Figure 2, Table 2). Results remained highly consistent across model 1 (B = 0.107, SE = 0.027, size = 312 mm2, pCFT = 0.001, adjusted for age, sex, and ethnicity), model 2 (B = 0.108, SE = 0.027, size = 312 mm2, pCFT = 0.001, additionally adjusted for child IQ and maternal education level), and model 3 (B = 0.110, size = 290 mm2, pCFT = 0.001, additionally adjusted for child behavioral problems), suggesting minimal residual confounding through various categories of covariates (Table 2). Results remained highly consistent when additionally adjusting model 3 using the broadband internalizing scale rather than the total problems scale (B = 0.107, SE = 0.027, size = 307 mm2, pCFT = 0.001). In addition, adjusting model 3 for exposure to traumatic life events (B = 0.107, SE = 0.027, size = 295 mm2, pCFT = 0.001) or for body mass index (B = 0.108, SE = 0.027, size = 279 mm2, pCFT = 0.001) did not change the results (Supplemental Table S2). A sex-by-target of bullying interaction term showed no significant clusters, suggesting the association is similar in boys and girls.


Figure 2 Images represent the left hemisphere clusters for perpetrators (left panel, view from medial side of the brain) and targets of bullying (right panel, view from inferior side of brain). Clusters represent areas which are different from children uninvolved in bullying (reference group). Models included one term for each of the three groups (perpetrators, targets, perpetrator–targets) all in the same model, and all reference coded to those uninvolved in bullying. S, superior, P, posterior, A, anterior, I, inferior, CFT, cluster-forming threshold. Red–yellow colors refer to thicker cortex, blue–light blue colors refer to thinner cortex.


Table 2 Results from whole-brain cortical thickness analyses.

In additional analyses utilizing a two-group (perpetrator or target of bullying) classification, results remained unchanged, suggesting the smaller perpetrator–target category did not influence results. When using a continuous sum score of victimization rather than categorical groupings, a similar cluster appeared in the fusiform gyrus, where high scores on victimization were related to thicker cortex (puncorrected = 0.0001, Supplementary Figure S1). However, this result did not remain after correction for the stringent multiple comparisons threshold. Lastly, no difference was observed in hippocampal or amygdala volume.


Children classified as perpetrators showed a thinner cortex in the cuneus at the “suggestive” threshold after correcting for multiple comparisons (pCFT = 0.005) but not at the more stringent threshold (pCFT = 0.001) and not fully adjusted for all covariates (i.e., model 3). Results remained consistent across the basic (model 1, B = −0.077, size = 501 mm2, pCFT = 0.005) and adjusted model (model 2, B = −0.084, size = 435 mm2, pCFT = 0.005), although disappeared when adjusting for child behavioral problems (Table 2). Additional analyses using a two-group classification (i.e., omitting the perpetrator–target category) showed consistent results. However, the cuneus cluster was not present when bullying involvement was examined continuously. Lastly, no difference was observed in hippocampal or amygdala volume.


No differences in cortical thickness were observed between perpetrator–targets and those uninvolved in bullying after correcting for multiple comparisons. Furthermore, no difference was observed in hippocampal or amygdala volume.


This large population-based study demonstrates differences in cortical morphology in children involved in bullying. Specifically, children identified as targets of bullying showed thicker cortex in the fusiform region compared to children uninvolved in bullying. The results demonstrate a new link between bullying involvement and structural brain morphology. Importantly, the results also provide an integral starting point for future work examining how cortical brain morphology relates to the persistent social and mental health problems that accompany those involved in bullying.

Children who were frequently victimized by perpetrators showed thicker cortex in the fusiform gyrus. This area, also part of Brodmann area 37, has been implicated in a wide array of functions, including facial and emotion processing, language, and theory of mind. Thicker cortex in this region could therefore be related to how targets of bullying perceive or recognize the faces of their aggressors. Interestingly, individuals with social anxiety disorder have been shown to exhibit differential neural activity to fearful as well as threatening faces (41, 42). A similar extension could be drawn to targets of bullying, where a sensitivity to certain facial expressions (e.g., angry/aggressive) could develop as a consequence of bullying. Alternatively, language ability has previously been proposed as a potential risk factor for targets of bullying, where children with underdeveloped language skills have been shown to be bullied more often (43, 44). As the fusiform gyrus has been implicated in aspects of verbal fluency (45), it is also possible that thicker cortex here represents a delayed development of language ability, which could in turn translate into a risk factor for bullying. Importantly, classification of targets of bullying was defined as being bullied once per week or more, which denotes frequent bullying involvement. When victimization was treated continuously rather than categorically, a similar cluster was observed, which suggests that such features of the fusiform may track into less frequently bullied children. Although the fusiform gyrus has been implicated in psychopathology (41), the current study was not able to determine whether it plays a mediating role in the development of psychopathology, as recent work has shown with other brain regions (21). Importantly, brain morphology linked to involvement in bullying may later manifest in other, more distant brain regions through atypical development of functional connectivity; such a downstream pathway may instead explain the persistent mental health and social problems experienced later in life.

At a conservative threshold for multiple testing correction, no differences were observed in cortical thickness in those classified as perpetrators. However, at a “suggestive” p value threshold and in models not adjusted for total psychiatric problems, a thinner cortex in the cuneus area was observed in those identified as perpetrators compared to those uninvolved in bullying. Part of the occipital lobe, the cuneus, is involved in various aspects of visual processing. A section of the cuneus has also often been implicated in the default mode network, one of the most commonly derived networks in resting-state functional MRI. Thus, future efforts should explore to what extent the default mode network is implicated in those involved in bullying. However, importantly, as this association was not significant in analyses where bullying was quantified continuously and was only observed at a relaxed correction for multiple testing, these results should be interpreted cautiously in the absence of external replication.

Interestingly, in children identified to be involved as perpetrator–targets, no differences were observed in cortical thickness. Given this group of children has the overall poorest prognosis, with higher rates of psychopathology and other problems later in life (25, 46, 47), this lack of difference in brain morphology is contrary to our a prior hypothesis. One potential explanation may lie in the sample size of this subgroup; it was the smallest group, with only 47 children, potentially limiting our ability to detect any differences. Conversely, there is likely considerable heterogeneity in any underlying morphological features in this group, suggesting that subtypes of bullying involvement may be important for brain development, or that other analytical methods may be necessary to detect differences (48).

Brain morphology has also been studied in the context of early life stress, trauma, maltreatment, and other experiences (4953). Differences across studies in terms of findings, methods used, and populations examined make summarizing the literature a challenge, although some interesting patterns emerge. Associations with the amygdala and hippocampus are certainly a common theme, although findings have been inconclusive (50, 5456). One central theory that aims to explain such deviations in brain development revolves around chronic stress exposure (57), and given the density of stress hormone receptors in the hippocampus, it may be particularly sensitive. Interestingly, no differences were found in this study in children who were targets of bullying. Timing of early-life exposures may be of particular relevance (14), which could explain why no association was found. Specifically, children may have been exposed to bullying behavior at varying times and durations, leading to heterogenous changes limbic brain structures that are difficult to detect, or may emerge later in life.

One important consideration of this study is the temporality of the brain–behavior relationship. As this study is based on a single neuroimaging assessment, it is not possible to delineate whether differences observed in cortical thickness develop before or after children become involved in bullying. In the context of targets of bullying, both scenarios are plausible. Targets of bullying could show differential fusiform development over time, either as an adverse consequence of bullying or even as a compensatory mechanism resulting from exposure to such behavior. Alternatively, such features of the fusiform gyrus could be present before the exposure to bullying, potentially acting as a predisposing factor. An example of such a mechanism can be found in the preceding paragraph discussing language ability; children with poor language abilities may be more prone to exposure to bullying. Future studies with longitudinal designs will allow for the determination of where on the neurodevelopmental trajectory they lie.

The established link between bullying involvement and persistent social and psychiatric problems later in life suggests the potential for a related and underlying neurobiological substrate. A similar construct has been proposed in the context of child maltreatment (10). Recent work has shown evidence for such a link via the basal ganglia (21). Alterations in the fusiform that potentially result from bullying could explain some facets of a given psychiatric disorder, for example altered cortical activity in individuals with anxiety disorder in response to emotional face processing (41) or emotionally valent images in individuals with depression (58). Importantly, brain alterations related to bullying involvement during childhood, which eventually co-occur with psychiatric sequela later in life, may require special consideration in future brain imaging research; the underlying neurobiology may be unique to bullying involvement and not necessarily common or etiological to the psychiatric symptomatology or overarching disorder. Such a phenomenon of an early life adversity leading to a particular brain alteration which co-occurs with psychiatric symptoms could explain some of the heterogeneity in the psychiatric neuroimaging literature and thus the lack of robust imaging biomarkers (59, 60).

Utilizing one of the world’s largest pediatric neuroimaging cohorts, we were able to examine the structural neural correlates of bullying involvement. Accompanying the power from this large sample size is the improved generalizability of the findings resulting from the population-based sampling, both in the reference group (those not involved in bullying) as well as in the groups exposed to bullying. Given the prospective and broad nature of the cohort, crucial information on potential confounding factors was also available. However, despite these clear strengths, some limitations warrant discussion. First, as described above, this study lacks repeated measurements of both bullying and brain imaging, and the assessment of bullying involvement takes place at a different age (i.e., before) than the MRI assessment. Longitudinal data will be crucial in delineating the precise temporal sequence of events and offer a crucial developmental perspective. Also of important note is the nonresponse analysis, which showed that the subsample children included in this study on average had slightly different characteristics compared with the full sample (e.g., 4 IQ points higher), suggesting some selection effects. Another broad issue in research on bullying involvement is related to how the data are characterized (e.g., continuous vs. categorically, frequency of involvement, etc.), which may impact results. Future work may also continue to explore latent constructs or latent classes of bullying involvement, which may offer additional insight by data-driven incorporation information (21). Lastly, this study relied on parent- and teacher-reported measures of bullying involvement, rather than child self-reports. Although parents and teachers have been shown to be reliable informants of bullying involvement, other strategies, such as peer nomination (61) and self-reporting, likely provide information with added value on bullying involvement, as victimization has been shown to be underreported in the absence of self-report (62).

This study demonstrates a link between bullying involvement and brain morphology in school-age children. In particular, children who are victimized by perpetrators have thicker cortex when compared to those uninvolved in bullying. These data offer evidence of disrupted cortical morphology in those involved in bullying and may offer cues to future work investigating the neurobiological underpinnings of associated and persistent problems later in life. Future work should utilize longitudinal neuroimaging data to concretely ascertain the different developmental trajectories involved.

Data Availability

The datasets generated and/or analyzed during the current study are not publicly available due to legal and ethical regulations, but may be made available upon request to the Director of the Generation R Study, Vincent Jaddoe (,, in accordance with the local, national, and European Union regulations.

Ethics Statement

The Medical Ethics Committee of the Erasmus Medical Center approved all study procedures, and all parents and children provided written informed consent and assent, respectively.

Author Contributions

RLM and HT developed the project and drafted the manuscript. RHM and SL collected data. RLM, RHM, SL, AC, TW, and HM conducted quality assurance of the data. RLM and SL conducted statistical analyses and developed software. RHM, SL, AC, PJ, BG, MV, MH, TW, and HM interpreted results and helped to draft and critically revise the manuscript. HT oversaw all aspects of the project.


This study was supported by the Sophia Foundation project S18-20 (awarded to R.L.M.). The Netherlands Organization for Health Research and Development (ZonMw) Vici project 016.VICI.170.200 (awarded to H.T.), and the Dutch Research Agenda (NWA) NeuroLabNL (400.17.602). Supercomputing resources were supported by the NWO Physical Sciences Division (Exacte Wetenschappen) and SURFsara (Cartesius compute cluster, HM was supported by the European Union’s Horizon 2020 research and innovation program (grant agreement 633595 and 733206). APCH was supported by the Netherlands Organization for Scientific Research. The general design of Generation R Study is made possible by financial support from the Erasmus Medical Center, Rotterdam, the Erasmus University Rotterdam, ZonMw, The Netherlands Organization for Scientific Research (NWO), and the Ministry of Health, Welfare, and Sport.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.


The Generation R Study is conducted by the Erasmus Medical Center in close collaboration with Faculty of Social Sciences of the Erasmus University Rotterdam, the Municipal Health Service Rotterdam area, Rotterdam, and the Stichting Trombosedienst & Artsenlaboratorium Rijnmond (STAR-MDC), Rotterdam. We gratefully acknowledge the contribution of children and parents, general practitioners, hospitals, midwives, and pharmacies in Rotterdam.

Supplementary Material

The Supplementary Material for this article can be found online at:


1. Craig W, Harel Y Bullying, physical fighting, and victimization. Young People’s Heal Context Int Rep from HBSC 2001/02 survery WHO Policy Ser Heal policy Child Adolesc issue 4, WHO Reg Off Eur. Copenhagen. (2004).

Google Scholar

2. Takizawa R, Maughan B, Arseneault L. Adult health outcomes of childhood bullying victimization: evidence from a five-decade longitudinal British birth cohort. Am J Psychiatry (2014) 171:777–84. doi: 10.1176/appi.ajp.2014.13101401

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Copeland WE, Wolke D, Angold A, Costello EJ. Adult psychiatric outcomes of bullying and being bullied by peers in childhood and adolescence. JAMA Psychiatry (2013) 70:419. doi: 10.1001/jamapsychiatry.2013.504

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Lereya ST, Copeland WE, Costello EJ, Wolke D. Adult mental health consequences of peer bullying and maltreatment in childhood: two cohorts in two countries. Lancet Psychiatry (2015) 2:524–31. doi: 10.1016/S2215-0366(15)00165-0

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Kumpulainen K. Psychiatric conditions associated with bullying. Int J Adolesc Med Health (2008) 20:121–32. doi: 10.1515/IJAMH.2008.20.2.121

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Moore SE, Norman RE, Sly PD, Whitehouse AJO, Zubrick SR, Scott J. Adolescent peer aggression and its association with mental health and substance use in an Australian cohort. J Adolesc (2014) 37:11–21. doi: 10.1016/j.adolescence.2013.10.006

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Stuart J E, Jose P. Is bullying bad for your health? The consequences of bullying perpetration and victimization in childhood on health behaviors in adulthood. J Aggress Confl Peace Res (2014) 6:185–95. doi: 10.1108/JACPR-01-2014-0003

CrossRef Full Text | Google Scholar

8. Gladden R, Vivolo-Kantor A, Hamburger M, Lumpkin C. Bullying surveillance among youths: uniform definitions for public health and recommended data elements. National Center for Injury Prevention and Control, Centers for Disease Control and Prevention and the United States Department of Education, Atlanta, GA (2013).

Google Scholar

9. Jansen PW, Verlinden M, Berkel AD, Mieloo C, van der Ende J, Veenstra R, et al. Prevalence of bullying and victimization among children in early elementary school: Do family and school neighbourhood socioeconomic status matter? BMC Public Health (2012) 12:494. doi: 10.1186/1471-2458-12-494

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Perry B. Incubated in terror: neurodevelopmental factors in the cycle of violence, Children. New York: Guilford Press (1997).

Google Scholar

11. Lenroot RK, Giedd JN. Brain development in children and adolescents: insights from anatomical magnetic resonance imaging. Neurosci Biobehav Rev (2006) 30:718–29. doi: 10.1016/j.neubiorev.2006.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Shaw P, Eckstrand K, Sharp W, Blumenthal J, Lerch JP, Greenstein D, et al. Attention-deficit/hyperactivity disorder is characterized by a delay in cortical maturation. Proc Natl Acad Sci U S A (2007) 104:19649–54. doi: 10.1073/pnas.0707741104

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hodel AS, Hunt RH, Cowell RA, Van Den Heuvel SE, Gunnar MR, Thomas KM. Duration of early adversity and structural brain development in post-institutionalized adolescents. Neuroimage (2015) 105:112–9. doi: 10.1016/j.neuroimage.2014.10.020

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Lupien SJ, McEwen BS, Gunnar MR, Heim C. Effects of stress throughout the lifespan on the brain, behaviour and cognition. Nat Rev Neurosci (2009) 10:434–45. doi: 10.1038/nrn2639

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Dahmen B, Puetz VB, Scharke W, von Polier GG, Herpertz-Dahlmann B, Konrad K. Effects of early-life adversity on hippocampal structures and associated HPA axis functions. Dev Neurosci (2018) 40(1):13–22 doi: 10.1159/000484238

PubMed Abstract | CrossRef Full Text | Google Scholar

16. du Plessis MR, Smeekens S, Cillessen AHN, Whittle S, Güroğlu B. Bullying the brain? Longitudinal links between childhood peer victimization, cortisol, and adolescent brain structure. Front Psychol (2019) 9:2706. doi: 10.3389/fpsyg.2018.02706

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Teicher MH, Samson JA, Sheu Y-S, Polcari A, McGreenery CE. Hurtful Words: Association of exposure to peer verbal abuse with elevated psychiatric symptom scores and corpus callosum abnormalities. Am J Psychiatry (2010) 167:1464–71. doi: 10.1176/appi.ajp.2010.10010030

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Crone EA, Konijn EA. Media use and brain development during adolescence. Nat Commun (2018) 9:588. doi: 10.1038/s41467-018-03126-x

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Will G-J, van Lier PAC, Crone EA, Güroğlu B. Chronic childhood peer rejection is associated with heightened neural responses to social exclusion during adolescence. J Abnorm Child Psychol (2016) 44:43–55. doi: 10.1007/s10802-015-9983-0

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Eisenberger NI. Does rejection hurt? An fMRI study of social exclusion. Science (2003) 302(5643):290–2. doi: 10.1126/science.1089134

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Quinlan EB, Barker ED, Luo Q, Banaschewski T, Bokde ALW, Bromberg U, et al. Peer victimization and its impact on adolescent brain development and psychopathology. Mol Psychiatry (2018) [epub ahead of print] doi: 10.1038/s41380-018-0297-9

CrossRef Full Text | Google Scholar

22. Kooijman MN, Kruithof CJ, van Duijn CM, Duijts L, Franco OH, IjMH van, et al. The Generation R Study: design and cohort update 2017. Eur J Epidemiol (2016) 31:1243–64. doi: 10.1007/s10654-016-0224-9

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Tiemeier H, Velders FP, Szekely E, Roza SJ, Dieleman G, Jaddoe VW, et al. The Generation R Study: a review of design, findings to date, and a study of the 5-HTTLPR by environmental interaction from fetal life onward. J Am Acad Child Adolesc Psychiatry (2012) 51:1119–1135 e7. doi: 10.1016/j.jaac.2012.08.021

CrossRef Full Text | Google Scholar

24. White T, Muetzel RL, El Marroun H, Blanken LME, Jansen P, Bolhuis K, et al. Paediatric population neuroimaging and the Generation R Study: the second wave. Eur J Epidemiol (2018) 33(1):99-125 doi: 10.1007/s10654-017-0319-y

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Lereya ST, Copeland WE, Zammit S, Wolke D. Bully/victims: a longitudinal, population-based cohort study of their mental health. Eur Child Adolesc Psychiatry (2015) 24:1461–71. doi: 10.1007/s00787-015-0705-5

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Rønning JA, Sourander A, Kumpulainen K, Tamminen T, Niemelä S, Moilanen I, et al. Cross-informant agreement about bullying and victimization among eight-year-olds: whose information best predicts psychiatric caseness 10–15 years later? Soc Psychiatry Psychiatr Epidemiol (2009) 44:15–22. doi: 10.1007/s00127-008-0395-0

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Schreier A, Wolke D, Thomas K, Horwood J, Hollis C, Gunnell D, et al. Prospective study of peer victimization in childhood and psychotic symptoms in a nonclinical population at age 12 years. Arch Gen Psychiatry (2009) 66:527–36. doi: 10.1001/archgenpsychiatry.2009.23

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Zwierzynska K, Wolke D, Lereya TS. Peer victimization in childhood and internalizing problems in adolescence: a prospective longitudinal study. J Abnorm Child Psychol (2013) 41:309–23. doi: 10.1007/s10802-012-9678-8

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Achenbach TM, McConaughy SH, Howell CT. Child/adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificity. Psychol Bull (1987) 101:213–32. doi: 10.1037//0033-2909.101.2.213

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Fischl B. FreeSurfer. Neuroimage (2012) 62:774–81. doi: 10.1016/j.neuroimage.2012.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Desikan RS, Segonne F, Fischl B, Quinn BT, Dickerson BC, Blacker D, et al. An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage (2006) 31:968–80. doi: 10.1016/j.neuroimage.2006.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Muetzel RL, Blanken LME, van der Ende J, El Marroun H, Shaw P, Sudre G, et al. Tracking brain development and dimensional psychiatric symptoms in children: a longitudinal population-based neuroimaging study. Am J Psychiatry (2018) appi.ajp.2017.1 175(1):54-62. doi: 10.1176/appi.ajp.2017.16070813

CrossRef Full Text | Google Scholar

33. Basten M, van der Ende J, Tiemeier H, Althoff RR, Rijlaarsdam J, Jaddoe VW, et al. Nonverbal intelligence in young children with dysregulation: the Generation R Study. Eur Child Adolesc Psychiatry (2014) 23:1061–70. doi: 10.1007/s00787-014-0551-x

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Achenbach TM, Rescorla LA. Manual for the ASEBA school-age forms & profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families. (2001).

Google Scholar

35. Dunn EC, Nishimi K, Neumann A, Renaud A, Cecil CAM, Susser ES, et al. Time-dependent effects of exposure to physical and sexual violence on psychopathology symptoms in late childhood: in search of sensitive periods in development. J Am Acad Child Adolesc Psychiatry (2019) doi: 10.1016/j.jaac.2019.02.022

CrossRef Full Text | Google Scholar

36. R Core Team. R: A Language and Environment for Statistical Computing. (2014).

Google Scholar

37. Hagler DJ, Saygin AP, Sereno MI. Smoothing and cluster thresholding for cortical surface-based group analysis of fMRI data. Neuroimage (2006) 33:1093–103. doi: 10.1016/j.neuroimage.2006.07.036

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Greve DN, Fischl B. False positive rates in surface-based anatomical analysis. Neuroimage (2018) 171:6–14. doi: 10.1016/j.neuroimage.2017.12.072

PubMed Abstract | CrossRef Full Text | Google Scholar

39. van Buuren S. Groothuis-Oudshoorn K: mice: multivariate imputation by chained equations in R. J Stat Softw Artic (2011) 45:1–67. doi: 10.18637/jss.v045.i03

CrossRef Full Text | Google Scholar

40. Rubin D. Multiple imputation for nonresponse in surveys. New York, NY: John Wiley and Sons (2004).

Google Scholar

41. Frick A, Howner K, Fischer H, Kristiansson M, Furmark T. Altered fusiform connectivity during processing of fearful faces in social anxiety disorder. Transl Psychiatry (2013) 3:e312. doi: 10.1038/tp.2013.85

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Etkin A, Wager TD. Functional neuroimaging of anxiety: a meta-analysis of emotional processing in PTSD, social anxiety disorder, and specific phobia. Am J Psychiatry (2007) 164:1476–88. doi: 10.1176/appi.ajp.2007.07030504

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Knox E, Conti-Ramsden G. Bullying risks of 11-year-old children with specific language impairment (SLI): does school placement matter? Int J Lang Commun Disord (n.d.) 38:1–12. doi: 10.1080/13682820304817

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Lindsay G, Dockrell JE, Mackie C. Vulnerability to bullying in children with a history of specific speech and language difficulties. Eur J Spec Needs Edu (2008) 23:1–16. doi: 10.1080/08856250701791203

CrossRef Full Text | Google Scholar

45. Abrahams S, Goldstein LH, Simmons A, Brammer MJ, Williams SCR, Giampietro VP, et al. Functional magnetic resonance imaging of verbal fluency and confrontation naming using compressed image acquisition to permit overt responses. Hum Brain Mapp (2003) 20:29–40. doi: 10.1002/hbm.10126

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Shetgiri R. Bullying and victimization among children. Adv Pediatr (2013) 60:33–51. doi: 10.1016/j.yapd.2013.04.004

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Kim YS, Leventhal BL, Koh Y-J, Hubbard A, Boyce WT. School bullying and youth violence. Arch Gen Psychiatry (2006) 63:1035. doi: 10.1001/archpsyc.63.9.1035

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Nenning K-H, Liu H, Ghosh SS, Sabuncu MR, Schwartz E, Langs G. Diffeomorphic functional brain surface alignment: functional demons. Neuroimage (2017) 156:456–65. doi: 10.1016/j.neuroimage.2017.04.028

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Teicher MH, Dumont NL, Ito Y, Vaituzis C, Giedd JN, Andersen SL. Childhood neglect is associated with reduced corpus callosum area. Biol Psychiatry (2004) 56:80–5. doi: 10.1016/j.biopsych.2004.03.016

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Woon FL, Hedges DW. Hippocampal and amygdala volumes in children and adults with childhood maltreatment-related posttraumatic stress disorder: a meta-analysis. Hippocampus (2008) 18:729–36. doi: 10.1002/hipo.20437

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Cohen RA, Grieve S, Hoth KF, Paul RH, Sweet L, Tate D, et al. Early life stress and morphometry of the adult anterior cingulate cortex and caudate nuclei. Biol Psychiatry (2006) 59:975–82. doi: 10.1016/j.biopsych.2005.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Nemeroff CB. Paradise lost: the neurobiological and clinical consequences of child abuse and neglect. Neuron (2016) 89:892–909. doi: 10.1016/j.neuron.2016.01.019

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Hart H, Rubia K. Neuroimaging of child abuse: a critical review. Front Hum Neurosci (2012) 6:52. doi: 10.3389/fnhum.2012.00052

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Hanson JL, Nacewicz BM, Sutterer MJ, Cayo AA, Schaefer SM, Rudolph KD, et al. Behavioral problems after early life stress: contributions of the hippocampus and amygdala. Biol Psychiatry (2015) 77:314–23. doi: 10.1016/j.biopsych.2014.04.020

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Krugers HJ, Arp JM, Xiong H, Kanatsou S, Lesuis SL, Korosi A, et al. Early life adversity: lasting consequences for emotional learning. Neurobiol Stress (2017) 6:14–21. doi: 10.1016/j.ynstr.2016.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Calem M, Bromis K, McGuire P, Morgan C, Kempton MJ. Meta-analysis of associations between childhood adversity and hippocampus and amygdala volume in non-clinical and general population samples. NeuroImage Clin (2017) 14:471–9. doi: 10.1016/j.nicl.2017.02.016

PubMed Abstract | CrossRef Full Text | Google Scholar

57. McEwen BS. Brain on stress: how the social environment gets under the skin. Proc Natl Acad Sci (2012) 109:17180–5. doi: 10.1073/pnas.1121254109

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Groenewold NA, Opmeer EM, de Jonge P, Aleman A, Costafreda SG. Emotional valence modulates brain functional abnormalities in depression: evidence from a meta-analysis of fMRI studies. Neurosci Biobehav Rev (2013) 37:152–63. doi: 10.1016/j.neubiorev.2012.11.015

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Venkatasubramanian G, Keshavan MS. Biomarkers in psychiatry—a critique. Ann Neurosci (2016) 23:3–5. doi: 10.1159/000443549

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Linden DEJ. The challenges and promise of neuroimaging in psychiatry. Neuron (2012) 73:8–22. doi: 10.1016/j.neuron.2011.12.014

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Verlinden M, Veenstra R, Ringoot AP, Jansen PW, Raat H, Hofman A, et al. Detecting bullying in early elementary school with a computerized peer-nomination instrument. Psychol Assess (2014) 26:628–41. doi: 10.1037/a0035571

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Shemesh E, Annunziato RA, Ambrose MA, Ravid NL, Mullarkey C, Rubes M, et al. Child and parental reports of bullying in a consecutive sample of children with food allergy. Pediatrics (2013) 131:e10–7 doi: 10.1542/peds.2012-1180

PubMed Abstract | CrossRef Full Text | Google Scholar

Keywords: cortical thickness, victimization, vertex-wise analysis, population based, fusiform

Citation: Muetzel RL, Mulder RH, Lamballais S, Cortes Hidalgo AP, Jansen P, Güroğlu B, Vernooiji MW, Hillegers M, White T, El Marroun H and Tiemeier H (2019) Frequent Bullying Involvement and Brain Morphology in Children. Front. Psychiatry 10:696. doi: 10.3389/fpsyt.2019.00696

Received: 13 April 2019; Accepted: 28 August 2019;
Published: 24 September 2019.

Edited by:

Tracy Vaillancourt, University of Ottawa, Canada

Reviewed by:

David Matuskey, Yale University, United States
Edward D. Barker, Birkbeck, University of London, United Kingdom

Copyright © 2019 Muetzel, Mulder, Lamballais, Cortes Hidalgo, Jansen, Güroğlu, Vernooiji, Hillegers, White, El Marroun and Tiemeier. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Henning Tiemeier,

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.