Good things peak in pairs: a note on the bimodality coefficient

Frequency data of four hypothetical distributions of 100 values each, with corresponding estimates of skewness (m3), kurtosis (m4), and the BC.


DISTRIBUTION ANALYSES AND BIMODALITY
Distribution analyses are becoming increasingly popular in the psychological literature as they promise invaluable information about hidden cognitive processes (e.g., Ratcliff and Rouder, 1998;Ratcliff et al., 1999;Wagenmakers et al., 2005;Miller, 2006;Freeman and Dale, 2013). One particular approach probes distributions for uni-vs. bi-modality, because bimodality often results from the contribution of dual processes underlying the observed data (Larkin, 1979;Freeman and Dale, 2013; see Knapp, 2007, for a historical overview). Although several statistical tools for this purpose exist, it remains unclear which one can be considered as a gold standard for assessing bimodality in practice. Freeman and Dale (2013) have recently shed some light on the utility of three different measures of bimodality known as the bimodality coefficient (BC; SAS Institute Inc, 1990), Hartigan's dip statistic (HDS; Hartigan and Hartigan, 1985), and Akaike's information criterion (AIC; Akaike, 1974) as applied to onecomponent and two-component Gaussian mixture distribution models (McLachlan and Peel, 2000). Overall, their analyses favored the HDS but also credited the BC with considerable utility. Notably, however, rather different formulas for the BC can be found in the literature (SAS Institute Inc, 1990, 2012Knapp, 2007;Bimodal distribution, 2013;Freeman and Dale, 2013)-certainly a potential source of confusion among researchers using the BC. 1 Additionally, the Appendix of Freeman and Dale (2013) gives a slightly ambiguous formula for the BC because their approach used non-standard MATLAB functions that are not widely accessible. The present article aims at clarifying and correcting these issues in an attempt to prevent misunderstanding and confusion. Further, methodological issues in using this measure are sketched to provide an intuition about its behavior. Note that the current paper does not intend to argue in favor of the BC as compared to other measures (see Freeman and Dale, 2013, for a thorough comparison). Rather, we want to point out pitfalls and limitations of this measure that can easily be overlooked.

THE BC AND ITS CAVEATS
The computation of the BC is easy and straightforward as it only requires three numbers: the sample size n, the skewness of the distribution of interest, and its excess kurtosis 2 (see DeCarlo, 1997, andJoanes andGill, 1998, for a detailed description of the latter two statistics). First appearing as part of the SAS procedure CLUSTER under the headline 1 The corresponding Wikipedia article (Bimodal distribution, 2013) used a wrong formula throughout, but has been corrected as part of preparing this article. 2 Excess kurtosis and Pearson's original kurtosis differ only as to whether the distribution's fourth scaled moment is normalised to a value of 0 for normal distributions or not (with "excess" indicating that a value of three has been subtracted for normalisation). The present article assumes all statistics to represent excess kurtosis if not explicitly indicated otherwise.
"Miscellaneous Formulas" of the SAS User's Guide (SAS Institute Inc, 1990, p. 561), the original formulation of the BC is , with m 3 referring to the skewness of the distribution and m 4 referring to its excess kurtosis (see Knapp, 2007, for critical remarks about this notation), with both moments being corrected for sample bias (cf. Joanes and Gill, 1998). The BC of a given empirical distribution is then compared to a benchmark value of BC crit = 5/9 ≈ 0.555 that would be expected for a uniform distribution; higher numbers point toward bimodality whereas lower numbers point toward unimodality.
Freeman and Dale (2013)  Irrespective of these computational issues, the above-mentioned formula reveals that the BC is directly influenced by both, skewness and kurtosis: Higher BCs result from high absolute values of skewness and low or negative values of kurtosis. Especially the influence of skewness can result in undesired behavior of the BC. As an illustration, four hypothetical distributions of 100 values each (range 1-11) are plotted in Figure 1, including their skewness, kurtosis, and the resulting BC (see Appendix for the raw data).
Comparing distribution A and B reveals the expected behavior of the BC: The two obvious modes in distribution B decrease kurtosis and increase the BC. Distribution C, however, is clearly unimodal when inspected by eye but its heavy skew also leads to a large BC. In terms of the BC, distribution C is even more bimodal than distribution D even though distribution D clearly has two modes, but otherwise both are very similar. The additional mode, however, decreases skewness thereby lowering the BC as long as it is not compensated by (negative) kurtosis.

CONCLUSIONS
As described above, empirical values of BC > 0.555 are taken to indicate bimodality. A probability density function for the BC, however, cannot be derived (Knapp, 2007). This is a major drawback because it precludes a thorough null-hypothesis significance test.
A suitable alternative test for bimodality is the dip test  that probes for deviations from unimodality (see also Freeman and Dale, 2013, for a more detailed description). An algorithm for this test was proposed after its publication (Hartigan, 1985) and this algorithm has meanwhile been adopted for MATLAB (Mechler, 2002). Additionally, an up-to-date, bug-corrected version was recently published as an R package (diptest, Maechler, 2012).
A direct comparison of the BC and the dip test  revealed that both measures have merit for assessing bimodality but neither statistic is perfectly sensitive and specific at the same time. Accordingly, one may assess empirical distributions with both measures and diagnose bimodality especially in case of convergent results. Should the results not converge, it seems the best strategy to investigate distributions for other measures, such as skewness and kurtosis individually (as well as their appearance when inspected by eye), to determine whether the result of the BC might be biased in one or the other direction.

ACKNOWLEDGMENTS
We are grateful to Ed Huddleston of the SAS Institute Inc. for providing detailed information about the evolution of the BC.