Consequences of inconsistently classifying woodland birds

There is a longstanding debate regarding the need for ecology to develop consistent terminology. On one hand, consistent terminology would aid in synthesizing results between studies and ease communication of results. On the other hand, there is no proof that standardizing terminology is necessary and it could limit the scope of research in certain fields. This article is the first to provide evidence that terminology can influence results of ecological studies. We find that researchers are classifying ‘woodland birds’ inconsistently because of their research aims and linguistic uncertainty. Importantly, we show that these inconsistencies introduce a systematic bias to results. We argue that using inconsistent terms can bias the results of studies, thereby harming the field of ecology, because scientific progress relies on the ability to synthesize information from multiple studies.


Introduction
Interpreting language is subjective and inexact. Language can be flexible and meaning is typically attributed to words on the basis of how people use them (Rey, 2005;Temmerman and Van Campenhoudt, 2011). However, sometimes this process results in omnibus terms which have too many meanings to be useful (Peters, 1991;Lindenmayer and Fischer, 2007). Using terms inconsistently can be problematic in scientific papers which must: (a) be replicable and; (b) convey a particular message as intended by the authors (Peters, 1991). From this need for exact terms comes the field of "terminology, " concerned with developing specific, universally-acknowledged terms (Pecman, 2014).
Terminologists propose that specialized fields like science, engineering and medicine require consistent language for clear and specific communication of findings (Cabré Castellví, 2003;L'Homme et al., 2003). This need for consistency is particularly pronounced in the medical field, due to the potentially high costs of miscommunication. This has led to the development of manuals guiding the use of medical terms (American Psychiatric Association, 2000;Gray, 2000).
In ecology, there have been sporadic efforts to promote consistency in terminology. In 1931, the Ecological Society of America formed the Committee on Ecological Nomenclature to promote consistent use of terminology by their members (Hanson et al., 1931). Since it disbanded in 1956, there has been little progress toward a consistent terminology in ecology (contra Laporte and Pey, 2014;Laporte and Garnier, 2012). Nevertheless, debate persists in the literature, with some arguing that inconsistent terminology is a problem in ecology (e.g., Peet, 1974;Herrando-Perez et al., 2014), and others suggesting that consistent terminology is unnecessary (Hodges, 2008(Hodges, , 2014Jax and Hodges, 2008).
Calls for the development of a consistent terminology in ecology focus on two primary issues (Mason and Langenheim, 1957;Hall et al., 1997;MacGregor-Fors, 2011). First, ambiguous terms are rife in ecology. Ambiguous terms are those which may be defined in several, often similar, ways (Regan et al., 2002). For example the "cover" of vegetation might be defined as foliage projective cover or crown cover, which are different in practice. Precise definition can reduce ambiguity. The second issue is one of vagueness in classification. Vague terms contain borderline cases in which it is hard to know whether something belongs in one category or another (Regan et al., 2002). For example, discrete categories such as "rare" or "widespread" are commonly used in ecology and related disciplines, but they group complex concepts into restrictive, often arbitrarily divided alternatives. The terms will be used inconsistently if researchers have different understandings of how categories should be distinguished (Jax, 2006). Inconsistent terminology can slow scientific progress (Herrando-Perez et al., 2014). It can lead to difficulties in finding and compiling relevant studies when reviewing the literature, which may cause redundant scientific investigations, or the inclusion of incomparable studies in a research synthesis (e.g., a meta-analysis) (MacGregor-Fors, 2011;Herrando-Perez et al., 2014). Inconsistent use of terminology can also cause problems when communicating findings to other scientists, policy makers and the public, as results may be misinterpreted or misrepresented due to ambiguous language (MacGregor-Fors, 2011;Herrando-Perez et al., 2014).
On the flipside, some researchers argue against the need for consistent terminology. Hodges (2008) suggests that there is no empirical proof that using terms inconsistently causes meaningful miscommunication in ecology. Furthermore, many authors define key terms in their articles, which minimizes ambiguity. Each individual article can be exact and unambiguous despite the lack of a consistent terminology between studies (Hodges, 2008). Another line of argument proposes that consistent terminology is unnecessary because the intended meaning is clarified by a term's context (Hodges, 2008;Araúz et al., 2013). Lastly, multiple definitions of a term can open a number of fields of enquiry which would have been proscribed by using one concrete definition. Therefore, a greater understanding of a problem might be achieved if it is less precisely defined (Hodges, 2008).
Regardless of arguments for and against a consistent terminology in ecology, there is a lack of empirical evidence to inform this debate. We address this gap by examining the classification of Australian woodland birds. Currently, no strong evidence exists that this group is being classified inconsistently (but see Reid, 1999;Kavanagh et al., 2007;Kinross and Nicol, 2008;Kinross and Nicol, for examples of studies which consider multiple classifications). Nevertheless, inconsistency in findings suggests that some unstudied variation might exist.
The majority of woodland bird articles cite a decline in woodland birds due to habitat destruction and degradation (Rayner et al., 2014a); however, the evidence of this is equivocal. Some studies show evidence of a decline in woodland birds (e.g., Barrett et al., 2004) while others do not (e.g., Rayner et al., 2014a). There is similar disagreement about whether woodland bird prevalence relates to vegetation extent (Major et al., 2001;Mac Nally and Horrocks, 2002) or fragmentation (Radford et al., 2005;Amos et al., 2013). The disagreement between these studies could be attributed to regional differences (Polyakov et al., 2013), or differences in temporal (Yen et al., 2011) or spatial scale (Lindenmayer et al., 2010), but may also be symptomatic of underlying disagreement about exactly what constitutes a "woodland bird." Deciding how to classify woodland birds raises many questions including: Does this term refer to birds that occur in woodlands and how often do they have to reside in woodlands to count? Do we only include species that nest in woodlands or also those that forage there? What if species only need woodlands for part of their life cycle? The way authors address these questions is often unclear and the species they include as "woodland birds" will differ depending on the answers. Thus, we chose to study inconsistency in the classification of Australian woodland birds for three reasons. First, inconsistency seemed plausible given the lack of classification guidelines and contradictory findings about how woodland birds respond to their habitat. Second, a sufficient body of research existed to obtain meaningful results. Third, any inconsistency has potential policy consequences.
Here, we investigate (i) how consistently researchers are classifying woodland birds, (ii) why inconsistencies are occurring and, most importantly, (iii) how inconsistencies are affecting conclusions about woodland bird ecology and management. We expected that researchers would classify most of the same species as woodland birds, but disagree about a few species that are difficult to classify because they partially depend on woodland vegetation, and are therefore subject to vagueness, or their biology or behavior is poorly understood. We were uncertain whether we would find evidence of substantial differences in the results of studies attributable to the use of different classifications. If so, it would demonstrate that inconsistent classification is impeding researchers fully understanding the ecology and management of woodland birds and that there may be benefit in developing a consistent definition of what constitutes a woodland bird.

Investigating Inconsistency in Woodland Bird Classification
We conducted a systematic review of research about Australian woodland birds using two concurrent methodologies in August 2014. We performed a Google Scholar search using the search terms "woodland birds" and "Australia" and recorded the digital databases that host the first 100 papers. We then performed a thorough search of these databases (Elsevier, Wiley online library, Taylor and Francis, CSIRO, Springer, Royal Society Publishing, PLoS one and JSTOR). To complement this, we also searched the online repositories Scopus and Web of Science. The search terms used in both instances were: "Australia" AND any of (1) "woodland bird" OR (2) "woodland dependent" AND "bird" OR (3) "woodland" AND "bird." Articles not based in Australia and those focused on non-avian species or communities, nonwoodland habitats or single or pairs of species were removed. Information about the number of articles found and excluded at each step of this process is presented in Supplementary Material Data Sheet 1. The articles obtained through this search were broken into 5 categories for articles that: (1) mentioned but did not study woodland birds; (2) studied woodland birds but did not specify the species; (3) specified the species they regarded as woodland birds but gave no information about the species which were classed as non-woodland birds; (4) were about a specific group of woodland birds such as "declining woodland birds"; and (5) those which specified both woodland and non-woodland species. We were only interested in lists which included both woodland and non-woodland species to ensure that common birds were classified as woodland species no more frequently than less common species (as would be the case if we included studies which only specified woodland species). When multiple articles used the same dataset and method of classifying woodland species (e.g., Radford et al., 2005;Radford and Bennett, 2007;Garrard et al., 2012), only one list was retained. This search process yielded 38 lists of woodland birds.
For each species we calculated the number of lists containing the species and the percentage of these that classified the species as a woodland bird. This method is subject to variation when there are few records for a particular species. For example, if a species was reported in only one list, it can only have been classified as a woodland species in either 0 or 100% of studies, which is unlikely to represent the value that might be achieved if more authors considered the species. In order to reduce this variation, those species that were present in 10 or fewer species lists were excluded from further analysis. This left us with data on the classification of 165 species. The frequency distribution of the results was then plotted to illustrate the inconsistency in woodland bird classification.

Testing the Influence of Inconsistent Classification
To assess the consequences of inconsistent use of the term "woodland bird, " we investigated how variation in woodland bird classification affected the findings and interpretation of a published ecological study (Garrard et al., 2012). This study investigates the vulnerability of woodland bird species to habitat fragmentation. The response of woodland birds to habitat fragmentation was the subject of 33% of the articles returned in our systematic review and, as such, we expect that our investigation is broadly applicable to woodland bird research in Australia.
The methods used by Garrard et al. (2012) and the alterations made to investigate the effect of inconsistent classification are described in Supplementary Material Data Sheet 2. In essence, Garrard et al.'s (2012) study involved two steps. First, they estimated the dispersal ability of a particular set of woodland bird species based on their traits. They then investigated the relationship between this estimate of dispersal ability and the probability of species occurring in landscapes with varying levels of tree cover aggregation, a measure of how clumped together trees are in the landscape which is roughly the inverse of habitat fragmentation (Radford and Bennett, 2007). We investigated the sensitivity of Garrard et al.'s findings about the relationship between dispersal ability and response to tree cover aggregation by simulating the effect of choosing different sets species to represent woodland birds, based on the percentage of studies in which species were classified as woodland birds.
Some authors consider all species present in woodlands to be woodland birds, while others have more stringent criteria. Therefore, we analyzed the effect of increasingly stringent classification criteria on the response of the Garrard et al. (2012) model (Supplementary Material Data Sheet 2). We ran the model 9 times on different sets of species, one which included all species, and 8 representing frequency thresholds of 10, 20, 30. . . 80% to simulate the effect of becoming more selective about which species are included as woodland birds. So, the list based on the 80% frequency threshold only included those species that are classified as woodland birds in 80% or more of lists. The list based on the 70% frequency threshold included the same species, but also those that classified as woodland birds in 70-80% of lists, and so on. The results of each model run are not independent but further analyses show that this does not affect our overall result (Supplementary Material Data Sheet 3).
We examined how the estimate of the mean species response to tree cover aggregation for each of these nested subsets compared with the original estimate from Garrard et al. (2012) to determine how classification inconsistency might influence ecological inference. Garrard et al.'s study used a subset of data collected in an earlier study (Radford et al., 2005). In our study, we included all species found during the original survey; we estimated the median dispersal distance of the species not included by Garrard et al. (2012)

Reasons for Inconsistency in Woodland Bird Classification
We were interested in understanding the reasons for inconsistency in classification. In particular, we were interested in: (1) how well recognized lack of consistency was in the research community; (2) why researchers were classifying species differently; and (3) whether researchers thought inconsistent classification was problematic. We emailed the authors (n = 131) of the 109 papers collected in the systematic review to invite them to participate in our survey. They were presented with the findings of the systematic review and a value representing how consistent the lists they used in their studies were with other research. They were then asked a series of questions via SurveyMonkey (Survey Monkey Inc, 1999) regarding how they classify woodland birds and woodlands, and their views on why researchers may be classifying species differently (a copy of the survey is available in Supplementary Material Data Sheet 4). Authors received up to 4 emailed reminders to prompt them to fill in the survey. Of the 131 authors we contacted, 69 completed the survey, 31 responded to say that they were not involved in the woodland bird aspects of the study, and 31 did not respond. Survey questions came under four main headings: experience, beliefs about classification consistency, how woodland vegetation and birds are classified, and why the classifications are different between studies. The experience section was intended to allow us to exclude answers given by people who had never been involved in classifying woodland birds and therefore were unable to meaningfully answer some of the questions. In the section on beliefs about classification consistency, experts were asked whether they agreed that some research questions required different lists of woodland birds, and whether using a standardized list would be detrimental to answering certain research questions. The next section asked them to select the criteria that they used to distinguish woodland vegetation, and woodland birds. In the section about why classifications differ between studies, experts were asked to rate, on a scale of 1-10, how much they believe each option contributed to differences in the classification of woodland birds. In order to collect quantitative information, answers to the majority of questions were given via multiple choice options (n = 6) or Likert scales (n = 2). The options posed in the multiple choice questions were drawn from definitions of woodlands and woodland birds found in the systematic review articles.

Systematic Review
Of the articles reviewed, 7 mentioned but did not study woodland birds, 32 studied woodland birds but did not specify the species, 28 specified the species they regarded as woodland birds but gave no information about the species which were classed as nonwoodland birds, 15 were about a specific group of woodland birds such as "declining woodland birds, " and 38 specified both woodland and non-woodland species. These 38 lists formed the basis of the analyses. Only including studies specifying woodland and non-woodland species avoids confounding species that are "non-woodland birds" with those "woodland birds" which were absent from the survey.
In total, 165 bird species were recorded in at least 10 of the lists examined by this study. Excluding species that were in fewer than 10 lists tended to exclude more species from water-dependent and uncommon orders of birds (details about the classification of species are supplied in Supplementary Material Data Sheet 5). Of the 165 species, 8 were recorded as woodland birds in every list and 13 species were always classified as non-woodland birds. The remaining 144 species were inconsistently classified at least once (Figure 1).
The bimodal frequency distribution represented in Figure 1 indicates that there is agreement regarding the classification of a substantial proportion of species, but that this is not unanimous and there is little certainty in the classification of many species.

Effects of Inconsistency
The data collected by Radford et al. (2005) and used for Garrard et al. (2012) comprised 126 species. When all species found during the original field surveys were included in the analysis, the predicted effect of landscape aggregation on prevalence was 3.0 (Figure 2; 95% credible interval 2.1-3.9). This is substantially smaller than the effect size estimated by Garrard et al. (2012), who estimated the mean effect of habitat aggregation (variable br in Supplementary Material Data Sheet 2) to be 5.9 with 95% credible intervals of 4.2-8.2. The effect sizes estimated from the other subsets of data increased with increasing frequency thresholds such that the 80% threshold yielded an effect of 6.1 (95% credible interval 4.5-7.6; Figure 2).   These factors are broken into shrub, tree and other categories to give a clear indication of how important they each were for determining whether vegetation is "woodland." The number of respondents listing each factor is given as well as the number of respondents listing any factor within the shrub and tree categories.

Survey Results
In total, 69 woodland bird experts filled in the survey. All researchers acknowledged that they list woodland birds differently to each other but the vast majority (n = 55, 80%) believe that using different lists of woodland birds is not problematic for ecological research (item 9 of survey). In fact, 39 (of 68, 57%) experts believed that it would be problematic to have a unified list of woodland species because it would hinder the investigation of certain research questions (item 4). The survey identified vagaries in the definition of woodland vegetation as a key reason for inconsistency of woodland bird classification (items 6 and 7). Researchers variously listed between 1 and 9 factors that affected the way they classified woodland vegetation. The majority of responses (n = 67, 97%) included a characteristic of trees, with a lesser number specifying that their classification was based on ecological vegetation classes or other similar systems (n = 23, 33%) or characteristics of shrubs (Table 1). Very few experts (n = 2, 3%) considered soil properties when classifying woodlands.
Researchers listed between 1 and 9 factors as contributing to their classification of species as woodland birds (item 5). Of the 14 options, all were selected at least once. The majority (n = 60, 87%) of experts considered occurrence when classifying woodland birds, and a substantial number of experts based their classification on traits (n = 57, 83%) or used an authorized classification such as one used in a field guide or journal article (n = 39, 57%) ( Table 2). There was also widespread use of exclusion criteria such as whether a species was nocturnal or a water bird (n = 25, 36%).
Authors were asked to rate possible reasons for inconsistencies between researchers (item 7) and those that ranked the highest were A) "different ideas about how to determine whether species rely on woodland vegetation, " (B) "different aims of research" and (C) "regional differences in the behavior or habitat requirements" and (D) "regional differences in the distribution of species" (Figure 3). Uncertainty about the behavior or habitat requirements (G) and distribution of species (F) ranked relatively low.

Discussion
The need for ecological terms to be classified consistently is disputed in the literature, and this is reflected in the responses to our survey. Although the surveyed experts were aware that the term "woodland bird" was being used to represent different sets of species, almost all thought the inconsistency was not problematic (n = 55 of 69). Over half of experts (n = 39 of 68) felt that conforming to a consistent definition would inhibit their ability to answer certain research questions, although many (n = 29 of 68) did not. This belief is consistent with Hodges' These factors are classed by underlying orientation into 5 classes: occurrence based, authorized classification, trait based, based on habitat associations and on exclusion criteria. As in Table 1, the number of respondents listing each factor is given as well as the number of respondents listing any factor within the 5 classes.
(2008) assertion that retaining flexibility in definitions of terms can allow a subject to be more fully explored. The results of our study highlight the trade-off between using flexible terminology for woodland birds, allowing researchers to fully explore nuanced research questions, and using standardized terminology, which facilitates generalizing across studies and improves clarity in communication.
Given the lack of evidence for standardization and the benefits of retaining flexibility, it is unsurprising to find inconsistency in ecological terms such as "woodland bird." Prior to our study, Hodges (2008) suggested there was no compelling evidence that consistent terminology is necessary in ecology. However, our study provides evidence that using terms inconsistently can be problematic, both for ecological inference and conservation. In the case of woodland birds, results changed markedly depending on the way the term "woodland birds" is defined, hindering comparison or synthesis of findings from different studies (Peters, 1991;MacGregor-Fors, 2011;Herrando-Perez et al., 2014).
We demonstrated that results could vary substantially depending on which definition of woodland bird the author considers. In the model developed by Garrard et al. (2012), we found that the effect of tree aggregation on the occurrence of woodland birds varies substantially depending on the species included in the definition of a "woodland bird." This is a clear demonstration that results from different studies are not necessarily comparable.
Importantly, the variation in findings attributed to inconsistent classification may also have direct management implications. The belief that tree cover aggregation has little effect on "woodland bird" occurrence may alter where revegetation or reserves are located in the landscape. Furthermore, the Victorian temperate-woodland bird community is protected under the Flora and Fauna Guarantee Act (Victorian Government, 2013) but the species included in this list are only a subset of the species which authors frequently consider as woodland birds (Supplementary Material Data Sheet 6). Our results demonstrate that researchers use the term "woodland bird" to refer to substantially different sets of birds, and have variable ideas about how this group of species should be identified. Researchers variously categorized "woodland birds" depending on their occurrence, their traits, their habitat associations, on authorized classifications or by using exclusion criteria. This inconsistent classification is a result of both linguistic vagueness (it is unclear where on the continuum of dependence on woodlands that a species becomes a "woodland bird") and ambiguity (it may have more than one meaning, such as "birds that occur in woodlands, " "birds that do not occur outside woodlands" or "birds that nest in woodlands") (Regan et al., 2002). This variation is unrecognized in literature reviews (Bennett and Watson, 2011;Ford, 2011), meta-analyses (Maron et al., 2011;Rayner et al., 2014b) and management recommendations (Paton and O'Connor, 2010). Researchers are thus combining information from studies that consider widely different sets of species, confusing the understanding of woodland bird ecology and management, and inhibiting the generalizability of their results.
This brings us to our core questions: does the uncertainty surrounding the term "woodland bird" actually matter? In this case, and in ecology more broadly, is it necessary to consistently define terms to understand ecological relationships and avoid misunderstandings and difficulties in the development of metaanalyses, literature reviews and management recommendations? Is the trade-off between flexible and standardized definitions of terms worth making?
Our findings demonstrate that the magnitude of an effect can depend on how the term "woodland bird" is defined. If two different researchers conducted the same study using the same data but different definitions of "woodland bird, " they might develop incongruent or even contradictory results. This is problematic when attempting to understand woodland bird ecology or predict how they will respond to land management. Only a small subset of woodland bird research uses identical lists of "woodland birds, " so researchers must choose between including information from many studies (which risks FIGURE 3 | Ratings assigned to each of 7 reasons that researchers use different lists to classify woodland birds. (A) different ideas about how to determine whether species rely on woodland vegetation, (B) different aims of research, (C) regional differences in the distribution of species in woodland and non-woodland areas, (D) regional differences in the behavior or habitat requirements of species, (E) different ideas about what constitutes woodland vegetation, (F) uncertainty about the distribution of the species in woodland and non-woodland areas, (G) uncertainty about the behavior or habitat requirements of different species. Error bars represent the 95% confidence intervals of the estimates.
terminological differences confounding results) or only including studies which use the same definition and list of "woodland birds" (which risks excluding valuable insights from other studies).
However, it is possible that standardizing the definition of "woodland birds" would make it more difficult to answer important ecological questions, as reflected in the responses of 39 (57%) respondents. The empirical results of this study, which show that classifying woodland birds consistently is important, conflict with the opinions of those experts who believe that standardizing the term "woodland bird" is unnecessary and possibly detrimental.Clearly, there is a trade-off between using flexible definitions of terms and standardizing terms.
Our research provides some of the first empirical evidence that using terms inconsistently in ecology is problematic and needs to be resolved. More research is required to examine whether this effect is widespread but we propose that ecologists need to carefully consider whether they are using terms consistently. Contrary to the argument that inconsistency in classification is not an issue in ecology because terms are context-specific and well-defined by authors (Hodges, 2008), we found that many studies either do not define the term "woodland birds, " or define it incompletely. We support an approach which increases the transparency of woodland bird research by making overt the species which are considered "woodland birds, " and detailing why they were classified accordingly. Beyond this, we believe that developing standardized definitions of key terms is vital to ecological research and management. When it comes to classifying ecological groups (such as woodland birds) we need to develop terms that have management relevance and, ideally, are based on ecological theory and empirical data. However, we recognize that standardizing terminology may come at a cost in terms of the flexibility of research. Therefore, we propose that, when the research question would be impeded by a standardized terminology, researchers either avoid using the term by studying species individually or using quantitative estimates of traits and habitat associations, or present their results alongside results achieved with the standardized terminology. This would allow flexibility to be maintained but also retain generalizability between studies.