Associations Between Openness Facets, Prejudice, and Tolerance: A Scoping Review With Meta-Analysis

The personality factor of openness to experience, which encompasses curiosity, imagination, and a desire for new experiences, has been associated negatively with prejudice and positively with the closely related value of tolerance. While these relationships have been reviewed at the factor level, there has been no review of research at the lower facet level. This review aims to uncover the relationships between the facets of openness and the constructs of prejudice and tolerance. We conducted a preregistered scoping review with meta-analysis following the recommended guidelines from Joanna Briggs Institute. A total of 2,349 articles were reviewed, with 16 primary research articles (or 17 studies) meeting the criteria for inclusion. Aggregated effect sizes via random-effect meta-analysis revealed that all revised neuroticism-extraversion-openness personality inventory (NEO-PI-R) and international personality item pool (IPIP)-based facets of openness significantly predicted prejudice and tolerance. Out of the three measures [i.e., NEO-PI-R, IPIP-NEO, and honesty-humility, emotionality, extraversion, agreeableness, conscientiousness, and openness to experience personality inventory (HEXACO-PI), and the facets of openness examined], the NEO-PI-R facet of value was most strongly associated with prejudice. In contrast, the NEO-PI-R facet of aesthetics was the facet most strongly associated with tolerance. However, these results should be treated as preliminary in light of the small number of meta-analyzed studies and more primary research studies are needed to confirm the trends found in this review. This review represents the first step in the systematic investigation of the link between the facets of openness and components of prejudice and tolerance and contributes toward explaining prejudice and tolerance.


INTRODUCTION
Recent high-profile examples of prejudice, discrimination, and violence against ethnic minorities in the United States and elsewhere have reignited a global discourse on the causes of prejudice and possible solutions (Subbaraman, 2020). Prejudice, defined as a negative attitude toward others based on their social group membership (Allport, 1954), is prevalent worldwide (Duckitt, 2019). Prejudice manifests itself in various forms (Abrams, 2010;Liao et al., 2016). For instance, prejudice can be expressed explicitly (e.g., "I dislike immigrants"; Legault et al., 2007), implicitly (e.g., a strong latent reaction toward pairing negative words with immigrants; Legault et al., 2007), blatantly (e.g., "immigrants are generally not very intelligent"; Akrami et al., 2000), or subtly (e.g., "It is a matter of not trying hard enough, if immigrants would only try harder, they could be as well off as the locals"; Pettigrew and Meertens, 1995). Prejudice carries severe negative social implications. For instance, prejudice causes harm to the social fabric of society (like inciting intergroup hostility, reducing willingness to cooperate; Tropp, 2003;Noh et al., 2007;Williams, 2018) and causes devastating physical and mental health outcomes to the individuals who experience prejudice (like increased rates of cardiovascular disease, mortality, posttraumatic stress, substance abuse, depression, and suicidal attempts; Ashburn-Nardo et al., 2007;Harris et al., 2012;Paradies et al., 2015;Dover et al., 2020). This review offers an insight into the possible causes of prejudice by examining how prejudicial attitudes and values associated with tolerance associate with the personality factor of openness and its underlying facets.
Personality, however, is not thought to be entirely unidimensional. Instead, personality psychologists have a consensus that personality exhibits a hierarchical structure, where higher-level personality factors subsume lower-level personality facets (Figure 1; Digman, 1990;Goldberg, 1993;Judge et al., 2013). For example, the openness factor is placed on a higher hierarchy level while openness facets, such as fantasy, aesthetics, feelings, actions, ideas, and values, are placed on a lower level (Figure 1; McCrae et al., 2005). The hierarchical structure of personality is well-validated (Digman, 1997;Mount et al., 2005;DeYoung, 2006;Rushton and Irwing, 2008;Woo et al., 2014a), with a general agreement that that there is one general factor of personality at the broadest and the highest level (Rushton and Irwing, 2008), two meta-traits (where plasticity refers to the basic tendencies toward personal growth, while stability refers to the basic tendencies toward socialization; Digman, 1997), followed by the Big Five factors (i.e., openness, conscientiousness, extraversion, agreeableness, and neuroticism), then aspects (DeYoung et al., 2007;Mussel et al., 2011), and lastly the facets at the lowest level of the hierarchy (Figure 1; McCrae et al., 2005).
Proponents of hierarchically structured personality propose that the openness factor accounts for broad behavioral tendencies while lower-level openness facets account for more specific behavioral inclinations (Judge et al., 2013). Several researchers have argued that personality facets may afford higher explanatory potential than personality factors as they contain specific variance that accounts for individual differences beyond those of the common factor (Ashton, 1998;Paunonen et al., 1999;Elleman et al., 2020). A recent meta-analysis supported this claim where they found regression models with all narrow facets combined outperformed models with all broad factors combined in the prediction of workplace behaviors (Pletzer et al., 2020). Similarly, several studies found openness facets held stronger associations with specific behavioral outcomes than the broad openness factor (Griffin and Hesketh, 2004;Hastings and O'Neill, 2009;Woo et al., 2014b). For instance, task performance was found significantly correlated with the openness facets of values (r = 0.49, p < 0.01; Griffin and Hesketh, 2004), intellect (r = 0.17, p < 0.05; Griffin and Hesketh, 2004), and ingenuity (r = 0.15, p < 0.05; Woo et al., 2014b) but not with the broad openness factor (r = 0.07, n.s.; Griffin and Hesketh, 2004; r = 0.05, n.s.; Woo et al., 2014b). All these findings supported the claim that openness facets improve the predictive validities of openness on behavioral outcomes.
Although there is little dispute on the hierarchical representation of personality (Judge et al., 2013), personality psychologists have disagreed on the number of facets underlying each personality factor (Hogan et al., 1996;Cattell and Mead, 2008). For instance, there are currently more than 10 different personality measures of openness facets, which exhibit significant variability in the number of facets underlying the openness factor (Schwaba et al., 2020). Openness factor has been variously proposed to comprise three (intellect, imaginative-creative, and perceptive; Saucier and Ostendorf, 1999), four (creative, unconventional, inquisitive, and aesthetic appreciation; Lee and Ashton, 2004), five (creative-uncreative, inquisitiveuninquisitive, deep-shallow, individualistic-dependent, and perceptive-unobservant;Hofstee et al., 1992), and six facets (fantasy, aesthetics, feelings, actions, ideas, and values;McCrae et al., 2005). The key reason for the lack of consensus is because personality theorists had adopted different approaches in their conceptualization of the underlying facet structure of the openness factor (Glick and Fiske, 1996;Hough and Ones, 2001;Woo et al., 2014a). The approaches adopted by personality theorists can be categorized as either the questionnaire approach (where the factor analyses of similar measures derive the facet structure, e.g., NEO-PI-R; Costa and McCrae, 1992) or the lexical approach [where the empirical reduction of personalitydescribing adjectives derives the facet structure; e.g., humility, emotionality, extraversion, agreeableness, and conscientiousness and openness to experience personality inventory (HEXACO-PI) by Lee and Ashton (2004), and IPIP-based measures by Goldberg (1999); see Table 1 for the definition of the facets of openness within revised neuroticism-extraversion-openness personality FIGURE 1 | Hierarchical model of personality (Costa and McCrae, 1992;Digman, 1997;McCrae et al., 2005;DeYoung et al., 2007;Rushton and Irwing, 2008). The descriptors referring to each of the hierarchical levels are provided on the right of this figure.

Openness facet
Description Example item

NEO-PI-R/IPIP-based measures
Values (NEO-PI-R)/Liberalism (IPIP) Readiness to challenge authority and reexamine values "I believe that there is no absolute right or wrong" Aesthetics (NEO-PI-R)/Artistic Interests (IPIP) Appreciation of natural and artificial beauty "I see beauty in things that others might not notice" Feelings (NEO-PI-R)/Emotionality (IPIP) Awareness of own's own inner feelings "I feel others' emotions" Fantasy (NEO-PI-R)/Imagination (IPIP) Uses imagination to create an interesting inner world "I have a vivid imagination" Ideas (NEO-PI-R)/Intellect (IPIP) Willingness to consider new and unusual ideas "I am interested in abstract ideas" Actions (NEO-PI-R)/Adventurousness (IPIP) Eager to try new activities and experience new things "I prefer variety to routine"

HEXACO-PI
Aesthetic appreciation Appreciation of beauty in arts and in nature "I can spend a long time studying a painting that I like"

Inquisitiveness
Eager to experience all aspects of nature and human world "I enjoy looking at maps of different places" Creativity Preference for originality and innovative "I would enjoy creating a work of art" Unconventionality Willingness to accept the unusual "I like hearing about opinions that are very different from those of most people" Adapted from Costa and McCrae (1992), Lee and Ashton (2004), and Maples et al. (2014).
inventory (NEO-PI-R), HEXACO-PI, and international personality item pool (IPIP)-based measures]. These different approaches have resulted in many different conceptualizations of the openness factor and its constituent facets, further contributing to the status of the openness factor as one of the least understood personality constructs.
To better understand the openness and its constituent facets, several researchers have argued for more research studies into the criterion-related validity of the openness facets (e.g., Hastings and O'Neill, 2009;Judge et al., 2013;Woo et al., 2014b;Christensen et al., 2019;Schwaba et al., 2020). Accumulating evidence on the predictor-criterion validity of narrow facets facilitates the understanding of the facet-specific variance that is often masked by aggregating facet scores into factor scores (Pletzer et al., 2020). In addition, identifying the differential criterion-relations of openness facets unveiled the facet-level relationship between openness and the outcome variable. For instance, job stress has been found to be uncorrelated with openness factor but was negatively correlated with the openness facets of liberalism (r = −0.15, p < 0.05) and adventurousness (r = −0.21, p < 0.01), and positively correlated with the openness facets of imagination (r = 0.25, p < 0.01) and emotionality (r = 0.29, p < 0.01; Griffin and Hesketh, 2004). In this review, we attempt to review the current evidence of the predictive utility of the facets of openness in a very different domain: that of prejudice and tolerance.

Facets of Openness and Prejudice
Low openness individuals, characterized by black-or-white thinking style, intolerance of ambiguity, authoritarianism, dislike of change, and rejection of deviance from the social norms, have consistently been found to be more prejudiced than their more open-minded counterparts (Hodson and Dhont, 2015). Conversely, individuals high in openness have been consistently linked with lower prejudice (Flynn, 2005;Duriez and Soenens, 2006;Ekehammar and Akrami, 2007;Sibley and Duckitt, 2008;Sturmer et al., 2013). Sibley and Duckitt (2008) conducted a meta-analysis of 71 studies. They reported that the openness factor had the strongest association with prejudice toward low status and disadvantaged groups like illegal immigrants, African Americans, female, and Asian immigrants out of the Big Five factors. Specifically, high openness individuals were found to be less prejudiced toward low-status outgroups. These findings were corroborated by Crawford and Brandt (2019), where their meta-analysis found high openness individuals were less prejudiced toward mixed-status outgroups (e.g., Mormons, rich people, atheists, antigay activists, bankers, Evangelical Christians, and conservatives).
Although the openness factor has a consistent negative relationship with prejudice, the strength of this association does not appear to be consistent across measures. For instance, in their meta-analytic study, Sibley and Duckitt (2008) found a significant difference in the correlation between the openness factor and prejudice across different personality measures. In their study, the openness factor was strongly correlated with prejudice when measured with the NEO-PI-R (Costa and McCrae, 2008) but weakly associated with prejudice when measured with the Big Five Inventory (BFI; John and Srivastava, 1999). One possible explanation is that NEO-PI-R captured both the factor-level and facet-level variance of openness associated with prejudice, while the BFI only captured the factor-level variance of openness (Sibley and Duckitt, 2008). Another explanation is that each personality measure captures a different subset of openness facets that may or may not relate to prejudice. To understand the role of openness in prejudice and tolerance, we argue that examining the specific contributions of a wide range of openness facets, as operationalized by various leading personality measures, might provide a more nuanced understanding.

Facets of Openness and Tolerance
Tolerance, defined as a value orientation toward difference (Hjerm et al., 2019), offers social psychologists an avenue to examine positive intergroup relations and the positive aspects of intergroup perceptions (Butrus and Witenberg, 2013). However, tolerance research has been made complicated by the fact that many researchers have treated prejudice and tolerance as if they were opposite ends of a spectrum (Verkuyten and Slooter, 2007;Witenberg, 2007;Bambulyaka, 2011;Brandt et al., 2015;Rapp and Freitag, 2015), despite evidence that they are related to yet distinct constructs (Van der Noll et al., 2010;Crawford, 2014;van Zalk and Kerr, 2014;Miklikowska, 2015). For instance, van Zalk and Kerr (2014) found that although intolerance and prejudice significantly reduced over time from early to late adolescence, there was a significant difference in their developmental trajectories. They also found that while a decline in intolerance was associated with the decline in prejudice, a lower level of prejudice was not associated with a lower level of intolerance (van Zalk and Kerr, 2014). Their findings highlighted two points: (a) Tolerance and prejudice are separate constructs with different causal pathways, and (b) tolerance and prejudice are inter-related but not equivalent constructs. As van Zalk and Kerr (2014) argued, tolerance and prejudice were only moderately correlated (i.e., r = −0.45), which indicate that only 20.25% (i.e., r 2 ) or 45% (i.e., r) of the individual differences were shared between tolerance and prejudice (for a detailed discussion on the use of r 2 or r as a percent of determination, please refer to Ozer, 1985). Despite the moderate correlation, the tolerance construct contained a unique variance that is not explained by prejudice.
Neuroscience research has also linked prejudice and tolerance to different neural structures (Amodio, 2014). For example, prejudice was strongly associated with the emotional centers of the brain (e.g., the amygdala, the orbital frontal cortex, and the insula; Beer et al., 2008;Chekroud et al., 2014), whereas tolerance is strongly associated with the goal-directed and behavioral regulation centers of the brain (e.g., the lateral prefrontal cortex, the anterior cingulate cortex, and the lateral prefrontal cortex; Bartholow et al., 2006;Beer et al., 2008;Amodio, 2014). These findings supported the argument that prejudice and tolerance should be treated as separate constructs. Conceptually, some researchers suggested treating prejudice and tolerance as two different forms of intergroup attitudes, with prejudice conceptualized as a preconceived negative evaluation of outgroup members (with historical, cultural, and developmental roots; Hjerm, 2005), while tolerance is conceptualized as a developmentally advanced moral reasoning ability coupled with prosocial beliefs and an understanding of equalitarian principles (e.g., social equality and equal rights; Miklikowska, 2015). Several researchers have argued for more systematic conceptual and empirical differentiation between prejudice and tolerance to understand how prejudice and tolerance interrelate (Butrus and Witenberg, 2013;van Doorn, 2014).
Tolerance consists of a cognitive component (i.e., awareness of difference and recognition of the problems of social discrimination and injustice), affective component (feelings of empathy and optimism), and behavioral component (willingness to act toward welcoming and integrating outgroup members; Cote and Erickson, 2009). Given that tolerance is linked with the capacity to hold multiple perspectives and accept differing values, it is no surprise that the disposition toward open-mindedness (i.e., openness factor) has been associated with tolerance (Weatherford and Spokane, 2013;Ackermann and Ackermann, 2015;Han and Pistole, 2017;Saef et al., 2019;Sparkman et al., 2019). In general, the openness factor has been positively linked with political tolerance (willingness to grant political rights to outgroups; Freitag and Rapp, 2015;Oskarsson and Widmalm, 2016), religious tolerance (willingness to recognize alternative religious faith; Proctor and McCord, 2009), universalism values (motivational goal toward social justice, equality, world peace, and unity with nature; Hamer et al., 2019), universaldiverse orientation (attitude of awareness and acceptance of both the similarities and differences among social groups; Han and Pistole, 2017), cross-cultural exploration (willingness to engage in activities aimed to further understand foreign cultures; Sturmer et al., 2013), multiculturalism (ideological belief in recognizing and appreciating ethnic differences in society; Sparkman et al., 2019), and multi-cultural competency (possesses cultural knowledge, awareness of the power dynamics among different cultural groups, and skillful in using culturally appropriate speech; Weatherford and Spokane, 2013).
Conversely, the openness factor has been negatively related to political conservatism (the ideological belief that opposesany changes in social structure-, rejects uncertainty, and accepts status inequality Sibley et al., 2012), right-wing authoritarianism (adherence toward social norms and aggression toward others who challenged conventional norms; Leone et al., 2012;Sibley and Duckitt, 2013), dogmatism (unchangeable certainty about the truths of the beliefs of an individual and rejects all other beliefs; Batool and Akram, 2020), religious fundamentalism (a belief that there is only one true religion or one method of religious teaching, and views all other religions as evil or destructive; Proctor and McCord, 2009;Carlucci et al., 2011), and ethnocentrism (negative attitudes toward all groups other than the group of an individual; Hamer et al., 2019).

The Present Research
As described in the preceding sections, the openness factor has been consistently linked with prejudice and tolerance (Ekehammar and Akrami, 2007;Sibley and Duckitt, 2008;Sturmer et al., 2013;Han and Pistole, 2017;Sparkman et al., 2019). However, the mechanism by which the openness factor protects against prejudice and promotes tolerance is still relatively unknown. In line with the recent evidence that prejudice and tolerance are separate constructs, this scoping review with meta-analysis examines how prejudice and tolerance relate to the various facets of the fundamental personality factor of openness to experience. To the best of our knowledge, there has been no attempt to review the extent of this literature and consolidate these findings into a coherent picture of the facet-level relationships between openness and the constructs of prejudice and tolerance. This scoping review supplemented with the meta-analytical synthesis of these facet-level relationships will respond directly to this need.
A scoping review is a particular type of systematic review that scopes for a potentially large and diverse body of literature relating to a research topic (Arksey and O'Malley, 2005). It is especially important for knowledge synthesis when there is a lack of understanding of key concepts within a topic (e.g., the lack of conceptual and empirical differentiation between prejudice and tolerance) and when a concept of interest is of a complex or heterogenous nature (e.g., the facet-level structure of openness; Pham et al., 2014). The scoping review offers a preliminary classification and systematization of the extant literature by providing a descriptive presentation of what is known about the key concepts, highlights the dominant methodologies used within the current literature, and identifies existing knowledge gaps (Peterson et al., 2017). Consequently, a scoping review was conducted on the existing literature on the openness facets and their association with prejudice and tolerance constructs. In addition, this review is supplemented with random-effect metaanalysis, which allowed us to reliably determine the average effect sizes of the associations between the facets of openness and the constructs of prejudice and tolerance from the available research in this area. Dependency of observations (i.e., the same group of experimenters or the same sample contributing to several effect sizes within the same model) was corrected as per the guidelines provided by Viechtbauer (2010). This scoping review with metaanalysis aims to enhance the existing knowledge in three ways: (a) providing the first review of the relationships between the facets of openness with prejudice and tolerance, (b) identifying the dominant measures of openness utilized in the existing literature, and (c) identifying possible research gaps in the literature to aid in the planning of future research on openness, prejudice, and tolerance.

Planning Stage
Based on the guidelines recommended by the Joanna Briggs Institute for a scoping review, the review objectives, selection criteria, and extraction methods for this review were planned, specified, and documented in a protocol. The protocol was preregistered and published (https://osf.io/yw9g8/) before data collection to provide transparency and limit the occurrence of reporting bias (Peters et al., 2020). The protocol specified that only studies examining the relationships between one or more facets of openness and prejudice or tolerance were included. Studies must investigate a personality construct that is explicitly positioned on the facet-level of the hierarchical model of personality (Figure 1) and measure personality as being a facet of the openness factor. Subject to this requirement, research studies examining all models and measures of openness, prejudice, and tolerance are eligible for inclusion. There were no inclusionary or exclusionary criteria on participants or settings. That is, all studies that examined the relationships between the facets of openness and either prejudice or tolerance were included, regardless of the type of participants (e.g., university students and members of the general public) or the research context (e.g., geographic location and cultural setting). Only academic literature (i.e., journal articles, conference papers, dissertations, books, and book chapters) describing primary research was considered for inclusion in this review. This scoping review with meta-analysis considered experimental, quasi-experimental, and correlational study designs. Theoretical papers, reviews, and opinion papers were excluded. Finally, only articles that were published in English were included in the scoping review.

Search Strategy
The scoping review utilized a three-step search strategy, as recommended by Peters et al. (2020). The first author conducted an initial search on Scopus and Web of Science between July and August 2020 using the following search terms: Openness AND facets AND Prejudice OR discrimination OR tolerance. From this initial search, key articles were identified, and the title, abstract, and keywords of these articles were screened for additional relevant search terms. An automation tool was also used to identify relevant search terms (Word Frequency Analyser; Clark et al., 2020). Following an iterative process, the authors finalized on the following search terms: (prejudice OR discrimination OR toleran * OR intoleran * OR diversity OR attitude * OR religio * OR ideology) for prejudice or tolerance, and [(openness OR intellect OR "big five" OR "five factor") AND facet * ] for openness facet(s).
The main search using the finalized search terms was then conducted across three major databases (i.e., Scopus, Web of Science, and ProQuest), with the search string adapted to each database. A supplementary search of Google Scholar and PsyArXiv provided a further search of the relevant gray literature. A librarian liaison officer specialized in psychology was consulted and reviewed the search strategy in this stage. The last search examined the reference list of selected articles and relevant metaanalytic studies (e.g., Sibley and Duckitt, 2008;Crawford and Brandt, 2019) for articles relevant to the review questions. The reference list search identified an additional 72 articles. In total, 2,349 records were identified from our search strategy (for a detailed breakdown on the number of articles retrieved from each database, see Figure 2).

Screening and Data Extraction Stage
Study selection involved screening all articles across two stages: (a) title and abstract screening, followed by (b) full-text screening (for a flowchart on the screening and selection process, see Figure 2). Citation files of all articles were first imported into a web-based systematic review software (Rayyan; Ouzzani et al., 2016) to aid the screening process. Screening tools were developed in advance (see Supplementary Materials), as per established guidelines (Polanin et al., 2019), to help reviewers evaluate the articles consistently and reliably. To assess for interreviewer reliability, all reviewers (i.e., authors of the current study) pilot-tested the screening tools on a sub-sample of 20 abstracts and attained a 90% inter-reviewer agreement, which satisfies the 75% minimum requirement (Tricco et al., 2018;Polanin et al., 2019). At least two reviewers screened all articles at each of the two screening stages. Articles that failed to meet the inclusionary criteria were excluded. Any disagreements that arose between the reviewers were resolved through discussion and achievement of consensus. The screening stage identified 16 articles that satisfied our review objectives and met all inclusion criteria (see Figure 2 for the complete PRISMA-ScR diagram; Tricco et al., 2018).
Using a preregistered data extraction tool (https://osf.io/ yw9g8/), information pertinent to the review aims was extracted from the final set of 16 articles. The data extracted were as follows: author(s), year of publication, sample characteristics (sample size, age, gender, and sampling methods), country of research, personality measure used, type(s) of prejudice or tolerance examined, variables examined (i.e., name of the independent variables and dependent variables), measures used, and key findings. The data extraction tool was pilot tested on two articles by two reviewers (i.e., the first author and the last author of this paper). A high inter-reviewer agreement was achieved; there was no discrepancy in the information extracted from the two reviewers. The primary author extracted the remaining 14 articles. All extracted data were collated and stored using Microsoft Excel. The author(s), year of publication, participant demographics, country of research, personality measure used, outcome measure(s) used, and key findings are summarized in Table 2. All extracted results were categorized based on the relevance to the review aims and were summarized in Table 3. Where applicable, meta-analyses of the facet-level associations were conducted using the metafor (Viechtbauer, 2010) and robumeta (Fisher and Tipton, 2015) packages for R (R Development Core Team, 2015). Throughout our meta-analyses, we used random effects, with restricted maximum likelihood as the method for estimating random effects. In addition, because multiple effect sizes were sometimes taken from the same article (notably in Christopher et al., 2013), dependency was corrected for these effect sizes. That is, effect sizes taken from the same article were assigned to the same value of the random effect in the meta-analysis (for more details, see Konstantopoulos, 2011). Forest plots were created to present the summary effect sizes of the facet-level associations (Figures 3, 4). Two methods were used to assess for publication bias: (a) Egger's weighted regression method, which is suited for small sample meta-analyses (Egger et al., 1997), and (b) fail-safe N-test assessment among these meta-analyzed associations.

Study Demographics and Characteristics
Following the Joanna Briggs Institute framework for scoping review (Peters et al., 2020) and the inclusion criteria outlined above, a total of 16 articles were identified (Figure 2). That is, 14 peer-reviewed publications, one book chapter (Huxley et al., 2015), and one PhD dissertation (Averhart, 2012) were identified to be relevant to the review objective ( Table 2). The 16 articles contributed 17 separate studies ( Table 3) that investigated the facet level associations of openness and indices of prejudice or tolerance.

Not investigated
15. Thompson et al. (2002) Christopher et al. (2013) were corrected for dependency. The effect sizes of the two studies in Ekehammar and Akrami (2007) corrected for dependency. The effect sizes in Onraet et al. (2011) article were corrected for dependency.

Types of personality measure
No of studies (i.e., k)

Tolerance (k = 2)
Diversity attitude (2) Universalism value (1) of facets of openness were IPIP-based measures (k = 5, 29%) and the HEXACO-PI (k = 2, 12%). The dominant use of the NEO-PI-R/3 was found in both tolerance and prejudice studies ( Table 3). Four of the seven articles (57%) that examined tolerance used the NEO-PI-R/3 to assess the facets of openness, and 6 of the 11 articles (50%) that examined prejudice used the NEO-PI-R/3 to assess the facets of openness ( Table 3).

Facets of Openness and Prejudice
Eleven studies examined prejudice as the dependent variable ( Table 3). All 11 studies used self-report questionnaires to measure the indices of prejudice; none of the studies include implicit or behavioral measures of prejudice ( Table 2). Out of the 11 studies, nine studies used established scales to assess prejudice, while two studies developed their prejudice measures (i.e., Proctor and McCord, 2009;Anglim et al., 2019). Together, these 11 studies examined a total of nine different types of prejudice (Table 3), namely: ageism (k = 2), sexual prejudice (k = 2), sexism (k = 3), racism (k = 2), ethnic prejudice (k = 1), mental disorder prejudice (k = 1), religious prejudice (k = 1), disability prejudice (k = 1), and generalized prejudice (k = 1). All studies used different measures to assess prejudice; no two studies used the same prejudice measure ( Table 2). Among studies that examined prejudice and used the NEO-PI-R/3 (k = 6; Table 3), the openness facet of values was consistently linked with all types of prejudice examined (i.e., sexism, generalized prejudice, sexual prejudice, and racism; Table 2 and Figure 3). Among studies that examined prejudice and used international personality item pool (IPIP)-based measures (k = 4), the facets of liberalism and artistic interests were associated with most indices of prejudice (Table 2 and Figure 3). Only one study examined prejudice and used the honesty-humility, emotionality, extraversion, agreeableness, and conscientiousness and openness to experience personality inventory (HEXACO-PI), and of the four indices of prejudice examined (i.e., sexism, racism, ageism, and disability prejudice), only the facet of aesthetic appreciation was associated with all indices of prejudice ( Table 2).

Facets of Openness and Tolerance
Seven studies examined tolerance as the dependent variable ( Table 3). All seven studies used self-report questionnaires to measure tolerance, and none of the studies included any implicit or behavioral measure ( Table 2). Most of the studies used established scales to assess tolerance, except for Kandler et al. (2012), who developed their tolerance measure. The seven studies examined five different types of tolerance, namely, universaldiverse orientation (UDO; k = 2), social equality orientation (k = 1), universalism value (k = 2), diversity beliefs (k = 1), and diversity attitudes (k = 1; Table 3).
Among the studies that examined tolerance and used the NEO-PI-R/3 (k = 4; Table 3), the openness facet of values was associated with all indices of tolerance (i.e., universal-diverse orientation, universalism value, and social equality orientation; Table 2 and Figure 4). The study that used IPIP-based measure found only the openness facets of liberalism, artistic interests, and emotionality to be associated with diversity beliefs ( Table 2 and Figure 4). Studies that examined tolerance and used the HEXACO-PI (k = 2) found all openness facets within HEXACO-PI (i.e., aesthetic appreciation, inquisitiveness, creativity, and unconventionality) to be associated with tolerance (i.e., diversity attitude and universalism value; Table 2).

Overall Effect Size (r) Aggregated Across Studies
Following the procedures outlined by Quintana (2015), we conducted a series of random effect meta-analyses on the associations of the facets of openness in NEO-PI-R and IPIP-based measures (see Figures 3, 4 for the forest plots). HEXACO facets of openness were excluded from this analytic strategy due to the limited number of studies. In addition, as the facets of openness in the IPIP-based measures (i.e., liberalism, artistic interests, emotionality, imagination, intellect, and adventurousness; Goldberg, 1999) were developed as proxy measures of the facets of openness in NEO-PI-R (i.e., values, aesthetics, feelings, fantasy, ideas, and actions), the facets of openness that were similar in theme were categorized together in the meta-analyses (see Table 1 for the detailed FIGURE 3 | Forest plots for the associations of the facets of openness with prejudice. The summary effect sizes are displayed as a polygon at the bottom of each plot with the width reflecting the 95% confidence interval of the average effect size estimate. The dotted line extending from the polygon reflects the 95% prediction interval which accounts for both the uncertainty of the effect size estimate and the uncertainty in the between-study variance estimate (Riley et al., 2011). Studies with larger squares contributed more to the summary effect sizes compared to the other studies. Studies with an asterisk (*) were potential outliers and influential cases as per standardized residual, Cook's distances, and hat values for each model. FIGURE 4 | Forest plots for the associations of the facets of openness with tolerance. The summary effect sizes are displayed as a polygon at the bottom of each plot with the width reflecting the 95% confidence interval of the average effect size estimate. The dotted line extending from the polygon reflects the 95% prediction interval which accounts for both the uncertainty of the effect size estimate and the uncertainty in the between-study variance estimate (Riley et al., 2011). Studies with larger squares contributed more to the summary effect sizes compared to the other studies. Studies with an asterisk (*) were potential outliers and influential cases as per studentized deleted residuals, Cook's distances, and hat values for each model.  Figure 3). Publication bias was assessed using Egger's (1997) regression test and Rosenthal's (1979) fail-safe N-test. Egger's regression tests of these associations were not significant, indicating a lack of bias (values/liberalism: z = −0.72, p = 0.47; aesthetics/artistic interests: z = −0.69, p = 0.49; feelings/emotionality: z = 1.18, p = 0.24; fantasy/imagination: z = 0.49, p =0.62; ideas/intellect: z = −0.13, p = 0.89; actions/adventurousness: z = 1.12, p = 0.26). According to Rosenthal's (1979) fail-safe N-test, a value that is five times greater than the number of studies included in the metaanalyses is needed to indicate a lack of publication bias (i.e., 60). The values of fail-safe N-test were 2,131 (values/liberalism), 536 (aesthetics/artistic interests), 645 (feelings/emotionality), 729 (fantasy/imagination), 495 (ideas/intellect), and 578 (actions/adventurousness) that all exceeded the criterion.
As our meta-analyses had shown, the facets of openness included in the NEO-PI-R/3 and its equivalent IPIP-based measures were all significantly associated with prejudice and tolerance. That is, the aggregated effect sizes of the relationship between each of these facets with prejudice and tolerance were all statistically significant (Figures 3, 4). Among the facets of openness in both the NEO-PI-R/3 and its equivalent IPIP-based measures, the facets of values (or liberalism), feelings (or emotionality), and fantasy (or imagination) were consistently linked with both prejudice (values/liberalism: r = −0.44; feelings/emotionality: r = −0.30; fantasy/imagination: r = −0.34) and tolerance (values/liberalism: r = 0.53; feelings/emotionality: r = 0.32; fantasy/imagination: r = 0.28). Our review also found specific facets of openness to be associated more strongly with indices of tolerance than with indices of prejudice. For instance, the aggregated correlation estimates found the facets of aesthetics (or artistic interests), ideas (or intellect), and actions (or adventurousness) were associated more strongly with tolerance (aesthetics/artistic interests: r = 0.53; ideas/intellect: r = 0.42; actions/adventurousness: r = 0.45) than prejudice (aesthetics/artistic interests: r = −0.27; ideas/intellect: r = −0.28; actions/adventurousness: r = −0.27).
Heterogeneity of the facet-level effect sizes was assessed using the Q and I 2 statistics. All Q statistics reached statistical significance, meaning that there is significant heterogeneity in the effect sizes of our included studies. In other words, the facet-level associations of openness with prejudice and tolerance do not share common effect sizes ( Table 4). The I 2 values, which indicate the percentage of total variation due to heterogeneity, ranged from 48.90 to 94.45%, meaning that most of the variations observed in the associations were largely due to between-study differences (Higgins et al., 2003). There are methods that could help identify the sources of heterogeneity in a meta-analysis (e.g., subgroup analysis and meta-regression; Song et al., 2001). However, due to the limited number of studies, these analyses were not conducted in this review. Sensitivity analyses (i.e., standardized residual, studentized deleted residuals, Cook's distances, and hat values; Viechtbauer and Cheung, 2010) also revealed the presence of outliers and influential studies. In sum, the large prediction interval of these facet-level associations (i.e., 95% prediction intervals of the associations with tolerance were all involved zero), the presence of outliers and influential cases, and the limited number of studies suggest that caution are warranted in the generalization of these aggregated correlation estimates.

DISCUSSION
The objectives of this review were to (a) comprehensively map the extant research on the relationships between facets of openness and prejudice/tolerance, (b) identify which facets have been most consistently linked with prejudice and tolerance, and (c) to further characterize the literature in terms of the specific openness measures used. To fulfill these objectives, this scoping review with meta-analysis adopted the systematic approach recommended by the Joanna Briggs Institute (Peters et al., 2020) and identified 16 relevant research articles. In this section, we first provide an overall summary of the review findings, followed by a discussion on the theoretical implications of specific findings. Lastly, we offer directions for future research and to note the limitations of this review.

Forms of Prejudice and Tolerance Represented in the Literature
The studies included in this review examined nine different forms of prejudice: ageism, racism, ethnic prejudice, sexual prejudice, sexism, mental disorder prejudice, religious prejudice, disability prejudice, and generalized prejudice (i.e., an aggregation of various forms of target-specific prejudice; Table 3). While these nine forms represent a reasonably broad cross-section of prejudice and discrimination literature (Duckitt, 1992;Sibley and Duckitt, 2008;Son Hing and Zanna, 2010), several prominent forms of prejudice (e.g., gender identity prejudice and antiimmigrant prejudice) have not yet been examined in terms of their relationships with openness facets. In addition, not all measures of prejudice were reviewed in our included studies. For example, none of our studies include an implicit measure of prejudice (e.g., the Implicit Association Test; IAT; Greenwald et al., 1998). Using implicit means to examine prejudice had been found to produce stronger indices of prejudice than those using self-report measures (e.g., Legault et al., 2007;Nosek et al., 2007). Some researchers regarded implicit measure of prejudice to be a more reflective measure of prejudice as participants are less able to control their responses to the measure and, hence, less likely to respond in a socially desirable manner (e.g., Cvencek et al., 2010;Geoffrey, 2013). Several studies had found participants who were motivated to suppress their prejudice (to avoid negative reactions from others) were more likely to report a lower prejudice score on self-report measures than implicit measures of prejudice (e.g., Devine et al., 2002;Legault et al., 2007). Therefore, it is likely that our included studies, which used only self-report measures of prejudice, captured only a thwarted level of prejudice, especially among participants who are motivated to suppress their prejudice for social desirability. Our review identified five different operationalization of the tolerance construct, i.e., universal-diverse orientation, social equality orientation, universalism value, diversity beliefs, and diversity attitudes ( Table 3). It is noteworthy that research into the psychology of tolerance began only recently in the 1990s in response to the growing tension between immigrants and the local citizens (Plaut, 2010). The relatively new field also meant that the characterization of the construct is still ongoing, and researchers did not consistently agree upon its operational definition. For instance, the definition for tolerance had ranged from (a) enduring things that one disliked or disapproved without interference (Van der Walt, 2014), (b) having a positive orientation toward minorities (Cote and Erickson, 2009), (c) adopting Egalitarian political ideology and attitude (Morley, 2003;Vasiljevic and Crisp, 2013;van Zalk and Kerr, 2014), (d) putting up with others for social harmony (Gibson, 1992;Vogt, 1994), and (e) embracing diverse ideas and opposing values . We argued that the lack of a clear and agreed-upon definition of tolerance is the key reason researchers have been unable to differentiate tolerance from prejudice. As found by our review, current literature operationalized the tolerance construct as either a (a) positive orientation toward differences (i.e., universal-diverse orientation, diversity beliefs, and diversity attitudes), (b) Egalitarian ideology (i.e., social equality orientation), or (c) universalism value (i.e., the pursuit of welfare and protection for all individuals). Several researchers have argued that Egalitarianism is not sufficient in promoting a truly tolerant society (Son Hing et al., 2008). Instead, researchers have advocated for tolerance to be operationalized as a positive orientation toward diversity, which is characterized by the awareness of intergroup differences, appreciation of diversity, and having a sense of relatedness toward outgroup members (Miville et al., 1999;Son Hing et al., 2008;Hjerm et al., 2019). As only seven studies on tolerance were retrieved, and only four studies operationalized tolerance as a positive orientation toward diversity (Table 3), it is recommended for future research to explore the operationalization of tolerance as an orientation toward diversity. More primary research studies on the association between the facets of openness and tolerance are also needed to broaden our understanding of the personality underpinning of this important aspect of tolerance.

Openness Facets and Their Relationships With Prejudice and Tolerance
Our review aimed to examine whether the relationships between the facets of openness and indices of prejudice and tolerance differ depending on how the openness facets were measured. Logically, this has to be the case since disagreement and uncertainty over the nature and the structure of openness (de Raad and van Heck, 1994;Hough and Ones, 2001;Woo et al., 2014a;Christensen et al., 2019) have resulted in different measures aligned with different theoretical perspectives. However, the small number of studies limited our ability to make systematized comparisons between personality measures. For instance, only one study examined prejudice using honesty-humility, emotionality, extraversion, agreeableness, conscientiousness, and openness to experience personality inventory (HEXACO-PI), and only one study examined tolerance using IPIP-based measures.
Within the identified studies, the NEO-PI-R and its variants were identified as the most frequently used measures of openness facets ( Table 3). Several studies have found that the NEO-PI-R/3 facets of openness mainly characterize the individual differences in openness toward non-intellectual experiences, such as aesthetic experiences, variety-seeking, daydreaming, and emotions (Table 1; e.g., Woo et al., 2014a;Christensen et al., 2019). While the NEO-PI-R/3 provides some coverage of individual differences in openness toward intellectual pursuits, such as intellectual curiosity toward ideas (i.e., the facet of ideas), several facets associated with intellectual pursuit are not captured (e.g., ingenuity, scientific curiosity, depth, and self-assessed intelligence; Woo et al., 2014a;Christensen et al., 2019). Therefore, reliance on the NEO-PI-R/3 represents a weakness of the extant literature, since investigations of facetlevel relationships between openness and prejudice do not examine the full breadth of the openness construct.
In our meta-analyses, we found the facets of values, feelings, and fantasy to be consistently linked with both prejudice and tolerance. The facets of feelings and fantasy represent the sensitivity and receptiveness of an individual toward inner processes (Albrecht et al., 2014). That is, individuals who scored high in the facet of feelings experience more differentiated emotional states (e.g., feeling the anxiety of meeting unfamiliar others while appreciating the excitement of meeting someone new), and individuals who scored high in the facet of fantasy are inclined toward creating a pleasant experiential and inner world while rejecting social norms and social expectations (Figure 1; Costa and McCrae, 1992;Connelly et al., 2014). These attributes of the facets of feelings and fantasy may explain how individuals high in these two facets could better appreciate differences (i.e., high in tolerance) while holding a pleasant attitude of diverse others (i.e., low in prejudice). The facet of values, which refers to the willingness of an individual to reexamine social, political, cultural, and religious values (Costa and McCrae, 1992), has also been positively linked with dispositional perspective taking (Miller, 2019) and negatively linked with right-wing authoritarianism (Sibley and Duckitt, 2010). Therefore, it may be argued that the facets of values, feelings, and fantasy promote tolerance and protect an individual against prejudiced attitudes via enhancing the ability to adopt the perspectives of others, resisting various forms of dogmatic and authoritarian attitudes, and increasing the motivation to shape pleasant intergroup relationships.
Our meta-analyses also found the facets of aesthetics, ideas, and actions to be linked more strongly with tolerance than prejudice. These facets (aesthetics, ideas, and actions) represent the sensitivity and receptiveness of an individual toward the external environment (Griffin and Hesketh, 2004). For instance, the individuals high in the facet of aesthetics are attuned toward appreciating beauty in their environment (e.g., natural, physical, and social environment), the individuals high in the facet of ideas are attuned to intellectual concepts (e.g., beliefs, worldviews, and philosophy), and individuals high in the facet of actions actively sought out new and unusual experiences (e.g., trying foreign foods, working in foreign countries, and traveling to exotic countries; Albrecht et al., 2014). The sensitivity toward the external environment and the receptiveness toward diverse experiences might influence tolerance more than prejudice. Similarly, the HEXACO-PI facet of unconventionality (i.e., the willingness to accept the new and unusual) were moderately associated with tolerance (universalism: r = 0.36, p < 0.001; diversity attitude: r = 0.16, p < 0.001) but weakly associated with prejudice (racism: r = −0.08, p < 0.05; sexism: r = −0.05, n.s; ageism: r = −0.05, n.s; see Table 2). Although these findings were derived from a limited number of studies, they do offer further support for the notion that prejudice and tolerance are related but separate constructs (e.g., Butrus and Witenberg, 2013;van Doorn, 2014). Using different openness measures and examining how the facets of openness relate separately to prejudice and tolerance may offer future researchers an avenue to differentiate prejudice and tolerance empirically and conceptually.
As this review has illustrated, the few studies investigating the associations between the facets of openness and prejudice or tolerance do not show entirely consistent findings (i.e., there is heterogeneity in the effect size estimates). Nevertheless, the outcomes of our random effects meta-analysis indicate statistically significant associations across all models; that is, none of the 95% confidence intervals of the average effect size estimates across all six facets had contained zero. The discrepancies in the results from included studies were likely related to the variation in the types of outcome measures used (e.g., the use of the original Miville Guzman Universality-Diversity scale or the short form version), the variation in the kinds of prejudice and tolerance examined (e.g., racism, sexism, and ethnic prejudice), or variation in the definition of the prejudice and tolerance examined (e.g., sexism as hostile, benevolent, or subtle). Other differences include the different cultures in which the research was conducted, the different age groups represented in the samples, and the different openness measures used. From examining the literature, it is clear that more research using consistent, broad measures of prejudice (e.g., a generalized prejudice measure), and tolerance (e.g., Miville Guzman Universality-Diversity scale) are needed to elucidate the relationship between the facets of openness with prejudice and tolerance. In addition, future studies looking to summarize this area of research may consider conducting subgroup analysis on potential moderators (such as the types of outcome measures and the types of openness measure) and examine how these moderators may affect the associations between facets of openness and prejudice and tolerance.

Lack of Non-Western Cultural Representation in the Current Literature
Almost all included studies were conducted using samples from Western cultures. Culture is a strong contextual factor that influenced the inner experience (e.g., how individual experiences and interprets social environment) as well as outward behaviors of an individual (e.g., how an individual behaves and interacts with others; Matsumoto et al., 2008;McDonald et al., 2011;Kende et al., 2018). Several studies had found contextual factors, such as culture, influenced both the development of personality (Allik and McCrae, 2004;Schmitt et al., 2007Schmitt et al., , 2008 and characteristic adaptations, such as intergroup behaviors and intergroup attitudes (Gerber et al., 2010;Kandler et al., 2012;Grijalva and Newman, 2015;Lee et al., 2018). Culture was also found to moderate the association of openness factor with intergroup bias and pro-diversity attitudes (e.g., Gerber et al., 2010;Alper and Yilmaz, 2019). For instance, the link between the openness factor and pro-diversity ideology is stronger among the participants from Western, educated, industrialized, rich, and democratic (WEIRD) cultures (Alper and Yilmaz, 2019). It is, therefore, very likely that contextual factors like culture moderate the relationship between the facets of openness and prejudice and tolerance. The facets of openness may influence prejudice and tolerance more strongly among participants from the WEIRD culture and weaker among participants from non-WEIRD culture. The relational pattern between the facets of openness and prejudice may also differ across cultures. The lack of cultural representation of non-Western settings represents a gap in the current literature and the generalizability of our findings. We also acknowledged that our search strategy of including only English articles likely inflated the numbers of WEIRD-biased samples. More research is needed to unravel the moderating effect of culture and examine the impact on the relationship between facets of openness and prejudice.

Limitations of the Present Research
Our key contribution is in providing a descriptive numerical summary of the diverse literature on the association of facets of openness with prejudice and tolerance. As the focus of a scoping review lies in contextualizing current knowledge (i.e., identifying the current state of understanding, methodologies used, and gaps in understanding; Levac et al., 2010), this scoping review with meta-analyses highlighted the diverse findings by categorizing studies and their conclusion into themes. Although measures were taken to ensure that all relevant articles were captured by our search strategy (i.e., recruiting a library liaison officer to develop a search string and searching for gray literature using Google scholar and preprint database), and three major databases were used in our search strategy, only articles that were written in English were selected. Our search strategy and selection criteria may have caused us to lose relevant articles released in other databases or written in languages other than English. Using only publications written in English also likely inflates the numbers of WEIRD-based studies from our search strategy.
Despite using a broad search strategy, only 16 articles were identified as relevant from the existing literature. It is emphasized that the limited number of articles identified in this review represents a weakness of the current literature in providing a proper systematization of the evidence on the predictorcriterion relationship between the facets of openness and different forms of prejudice and tolerance, and not necessarily represents a weakness of our search strategy. Nevertheless, the limited number of studies summarized in this review restricted the generalizability of our conclusion. That is, the relations between the facets of openness and the constructs of prejudice and tolerance outlined in this review should be treated as preliminary. More studies are needed to validate (or invalidate) the associations of facets of openness with indices of prejudice and tolerance summarized in this paper before any firm conclusions can be made.

Future Directions
Many psychologists have argued that using factor scores might obscure facet-criterion relationships (e.g., Hastings and O'Neill, 2009;Woo et al., 2014b). This review provides a more nuanced understanding of the facet-level associations between the openness factor and the different forms of prejudice and tolerance. As discussed, not all the facets of openness are associated equally with prejudice and tolerance. The associations reported in our review were based only on a few studies and thus should be treated as preliminary; future replication studies are needed to confirm the trends found in this review. As more evidence becomes available, an update in this review is encouraged to provide a timely guide for researchers interested in the link between openness facets and forms of prejudice and tolerance.
Recently, Woo et al. (2014b) had developed a new measure, the Six-Facet Openness Scale (SFOS), which covered most of the facets of openness unexamined by the NEO-PI-R. This new measure may complement NEO-PI-R in examining the full spectrum of openness facets. Other measures of the facets of openness beyond those discussed in this review may also be considered in future investigations to explore the broad range of openness facets and their relationship with prejudice and tolerance. For instance, the nine intellect scales in the Abridged Big Five Circumplex (AB5C; Hofstee et al., 1992), the homogenous item clusters of intellectance and school success in the Hogan Personality Inventory (Hogan et al., 1996), and the analytical item cluster in the Jackson Personality Inventory (Jackson, 1994) contain openness to intellectual pursuits not otherwise covered by the NEO-PI-R. For more definitive conclusions about the openness facets from different inventories and prejudice/tolerance, a large-scale study or series of studies in which all inventories assessing openness facets are included, and multiple kinds of prejudice and tolerance are assessed in different ways (including behavioral and implicit measures of prejudice), may be necessary.
More studies are also needed to examine the predictive utility of the facets of openness in prejudice and tolerance among non-Western cultures. The majority of the studies that examined the relationship between the facets of openness and prejudice or tolerance were conducted in western countries; no studies were identified from Africa, South America, and major parts of Asia (i.e., central, south, east, or south-east Asia). As discussed earlier, cultures might likely moderate the relationship between the facets of openness and prejudice. Alper and Yilmaz (2019) noted that the openness factor was strongly associated with Egalitarianism only when the presiding culture is liberal and does not oppress freedom of expression. Conversely, the openness factor was weakly linked with Egalitarianism when the culture was conservative and authoritarian. It is recommended for more studies to be conducted in non-Western countries (i.e., Africa, South America, and Asia) and for future studies to examine how different cultures (i.e., conservative cultures) affect the relationship between the facets of openness and prejudice or tolerance.

CONCLUSION
This scoping review with meta-analysis provides a preliminary guide on the link between the facets of openness and different indices of prejudice or tolerance. This review is also the first study that systematically reviewed the relationship between the facets of openness and tolerance. Several gaps in the current literature were identified; future studies looking into the relationship between the facets of openness and prejudice or tolerance may consider following our recommendation and building on the gaps identified. Ultimately, this review adds to the growing research in prejudice and tolerance and contributes knowledge toward identifying the personality of prejudice and tolerance.

AUTHOR CONTRIBUTIONS
DN created research questions, conducted the article search, prepared the raw data, screened and analyzed the data, and prepared the manuscript. PL and NM screened the research articles and contributed to manuscript editing. KC provided advice on the statistical methods (i.e., meta-analysis) and contributed to manuscript editing. JR created research questions, screened the research articles, and contributed to manuscript editing. All authors contributed to the article and approved the submitted version.

FUNDING
This research publication was funded by the Internal Research Fund, James Cook University, Singapore.

ACKNOWLEDGMENTS
We would like to thank Ms. Jayshree Mamtora from the Library and Information Services at James Cook University for her support in this review. We would also like to express gratitude to Dr. Daniel Quintana (2015) for making his R script publicly accessible for researchers who are new to R programming.