Impact Factor 3.560 | CiteScore 3.1
More on impact ›

REVIEW article

Front. Phys., 10 December 2020 | https://doi.org/10.3389/fphy.2020.604053

The Three Extreme Value Distributions: An Introductory Review

  • PoreLab, Department of Physics, Norwegian University of Science and Technology, Trondheim, Norway

The statistical distribution of the largest value drawn from a sample of a given size has only three possible shapes: it is either a Weibull, a Fréchet or a Gumbel extreme value distributions. I describe in this short review how to relate the statistical distribution followed by the numbers in the sample to the associate extreme value distribution followed by the largest value within the sample. Nothing I present here is new. However, from experience, I have found that a simple, short and compact guide on this matter written for the physics community is missing.

Introduction

Extreme value statistics offers a powerful tool box for the theoretical physicist. But it is the kind of tool box that is not missed before one has been introduced to it—perhaps a little like the smart phone. It concerns the statistics of extreme events and it aims to answer questions like “if the strongest signal I have observed over the last hour had the value x, what would the strongest signal expected to be if measured over hundred hours?” Furthermore, if I divide up this hundred-hour interval into a hundred 1-h intervals, what would be the statistical distribution of strongest signal in each 1-h interval?

It is the latter question which is the focus of this mini-review.

There is no lack of literature on extreme value statistics, see e.g., [15] or simply google the term. We find it used in connection with spin glasses and disordered systems [6], in connection with 1/f noise [7], in connection with optics [8], in connection with fracture [9] or the fiber bundle model [10], in diffusion processes [11] etc. There are plenty of examples from diverse fields of physics.

So, there is no lack of material for the novice that has seen a need for this tool. The problem is that it is not so easy to penetrate the literature, which is often cast in a rather mathematical language which takes work to penetrate. The aim of this mini-review is to present the theory behind and the main results concerning the extreme value distributions in a simple and compact way. We will present nothing new. For a longer, wider and more detailed review of extreme value statistics, Fortin and Clusel [12] or Majumdar et al. present exactly that [13].We have a statistical distribution p(x) and its associated cumulative probability

P(x)=xp(x)dx,(1)

which is the probability to find a number smaller than or equal to x. We draw N numbers from this distribution and record the largest of the N numbers. We repeat this procedure M times and thereby obtain M largest numbers, one for each sequence. What is the distribution of these M largest numbers in the limit when M, which then defines the extreme value distribution?

It turns out that depending on p(x), the extreme value distribution will have one of three functional forms:

• The Weibull cumulative probability

Φ(u)={e(u)αforu<0,1foru0,(2)

where we assume α>0. Note that Φ()=0. The corresponding Weibull extreme value distribution is

ϕ(u)={α(u)α1e(u)αforu<0,0foru0.(3)

• The Fréchet cumulative probability

Φ(u)={0foru0,euαforu>0.(4)

Also here we assume α>0. Note that Φ()=1. The Fréchet extreme value distribution is

ϕ(u)={0foru0,αuα1euαforu>0.(5)

• The Gumbel cumulative probability

Φ(u)=eeu,(6)

where <u<, so that Φ()=0 and Φ()=1. The corresponding Gumbel extreme value distribution is given by

ϕ(u)=eueu.(7)

The questions are 1. which classes of distributions p(x) lead to which of the three extreme value distributions and 2. what is the connection between x and u in each case? It turns out that.

• distributions where p(x)=0 for x>x0 and p(x)(x0x)α1 as xx0, see Eq. 10, lead to the Weibull extreme value distribution,

• distributions where p(x)xα1 as x, see Eq. 24 lead to the Fréchet extreme value distribution,

• and distributions where p(x) falls of faster than any power law as x, see Eq. 53 lead to the Gumbel extreme value distribution.

Furthermore, we will find that.

• for the Weibull extreme value distribution, u is given in terms of x in Eq. 13,

• for the Fréchet extreme value distribution, u given in terms of x in Eq. 27,

• for the Gumbel extreme value distribution, u is given in terms of x in Eqs 51 and 43.

We summarize these results in Table I.

TABLE 1
www.frontiersin.org

TABLE 1. Summary of main results.

The discussion that will now follow, will be built on the following relation. We draw N numbers from the probability distribution p(x): x1,x2,,xN. The probability that all the N numbers are smaller than or equal to a value x is

Prob[x1x,x2x,,xNx]=[xp(x)dx]N=P(x)N,(8)

where P(x) is the cumulative probability 1. Our task is to figure out the limit Prob[x1x,x2x,,xNx]=P(x)NΦ(u) as N, and what is u=u(x) as we approach this limit.

Rather than the conventional approach (see e.g., [10]) to this subject based on the Fréchet, Fisher and Tippett stability criterion [1], I will base the entire discussion on the relation

limN(1+xN)=ex.(9)

I believe this to be the simpler and more intuitive way.

Weibull Class

We consider here probability distributions p(x) having the form

p(x)={bα(x0x)α1forxx0,0forx>x0,(10)

where b is positive. We note that 0<α<1 leads to a diverging probability density as xx0. We furthermore note that α=1 implies that p(x) approach a constant when xx0 — which for example is the case when the distribution is uniform. The corresponding cumulative probability is given by

P(x)={1forxx0,1b(x0x)αforxx0.(11)

The extreme value cumulative probability for N samplings is given by

P(x)N=[1b(x0x)α]N,(12)

for xx0. We introduce the variable change

xx0=,u(bN)1/α(13)

where the reader should note that b is defined by the original distribution 10. Equation 12 then becomes

P(x)N=[1(u)αN]N.(14)

In the limit of N, this becomes

Φ(u)=limNP(x)N=e(u)α,(15)

for negative u. Hence, we have that

Φ(u)={e(u)αforu<0,1foru0,(16)

which is the Weibull cumulative probability, valid for all values of u even though we only know the behavior of p(x) close to x0. The Weibull probability density is given by

ϕ(u)=dΦ(u)du={α(u)α1e(u)αforu<0,0foru0.(17)

We note that the Weibull distribution resembles a stretched exponential. This is correct for α<1. However, α1 is much more common in the wild.

We express the Weibull cumulative probability in terms of the original variable x using Eq. 13,

Φ(u)=Φ((bN)1/α(xx0))=eNb(x0x)α=Φ˜(x).(18)

Hence, in terms of the original variable x, the Weibull extreme value distribution becomes

ϕ˜(x)=dΦ˜(x)dx=Nbα(x)α1eNb(x0x)α.(19)

Weibull: An Example

We now work out a concrete example. Let us assume that p(x) is given by

p(x)={0forx<0,α(1x)α1for0x1,0forx>1,(20)

i.e., b=1 and x0=1 in Eq. 10. The cumulative probability is then

P(x)={0forx<0,1(1x)αfor0x1,1forx>1.(21)

From Eq. 19 and we have that

ϕ˜(x)=Nα(1x)α1eN(1x)α.(22)

We show the distribution 20 with α=3 together with the corresponding extreme value distributions for N=100 and N=1,000, Eq. 19 in Figure 1A.

FIGURE 1
www.frontiersin.org

FIGURE 1. (A) The curve that has its maximum at x=0 is the probability distribution 20 with α=3. The curve that has its maximum in the middle is ϕ˜(x), Eq. 22 with N=100 and the curve that has its maximum to the right is ϕ˜(x) with N=1,000. (B) The histograms shown here are based on data according to the probability distribution 20 with α=3. The histogram having its maximum to the left shows all the generated data. The histogram having its maximum in the middle shows the largest number among each sequence of numbers of length 100, and the histogram having its maximum to the right shows the largest number among each sequence of numbers of length 1,000. We generated 107 sequences for both cases.

Using a random number generator producing IID numbers1r uniformly distributed on the unit interval, we may stochastically generate numbers that are distributed according to the probability density p(x) given in 20. We do this by inverting the expression P(x)=r, where the cumulative probability is given by 21. Hence, we have

x=1r1/α,(23)

where we have also used that r may be substituted for 1r in 21. We generate a sequence of sequences of numbers using this algorithm, each sequence having length N. We then identify the largest value within each sequence. We chose N=100 and N=1,000, in each case generating 107 such sequences. The histograms based on the random numbers themselves, and of the extreme values for each sequence of length either 100 or 1,000 we show in Figure 1B. This figure should be compared to Figure 1A.

The Weibull distribution, Eq. 17 is much used in connection with material strength [15]. This is no coincidence. Consider a chain. Each link in the chain can sustain a load up to a certain value, above which it fails. This maximum value is distributed according to some probability distribution. When the chain is loaded, it will be the link with the smallest failure threshold that will break first causing the chain as a whole to fail. Hence, the strength distribution of an ensemble of chains is an extreme value distribution, but with respect to the smallest rather than the largest value. The link strength must a positive number. Hence, the link strength distribution is cut off at zero or some positive value. The distribution close to this cutoff value must behave as a power law in the distance to the cutoff, e.g., due to a Taylor expansion around the cutoff. The corresponding extreme value distribution, which is the chain strength distribution, must then be a Weibull distribution.

Fréchet Class

We now assume that the probability distribution p(x) behaves as

p(x)=bαxα1forx,(24)

and the corresponding cumulative probability behaves as

P(x)=1bxαforx.(25)

The extreme value cumulative probability for N samplings is given by

P(x)N=[1bxα]N,(26)

for x. We introduce the variable change

x=(bN)1/αu,(27)

where b comes from the original distribution 24. We now plug this change of variables into Eq. 26 to find

P(x)N=[1b((bN)1/αu)α]N=[1uαN]N.(28)

In the limit of N, this becomes

Φ(u)=limNP(x)N=euα,(29)

where u0 is given by Eq. 27. We see that Φ(u)0 as u0+. Furthermore, for u<0, the function is no longer real. Hence, we define Φ(u)=0 for u<0. The ensuing extreme value cumulative probability is then given by

Φ(u)={0foru0,euαforu>0,(30)

which is the Fréchet cumulative probability. The Fréchet probability density is given by

ϕ(u)=dΦ(u)du={0foru0.αuα1euαforu>0.(31)

We express the Fréchet cumulative probability in terms of the original variable x using Eq. 27,

Φ(u)=Φ(x(bN)1/α)=eNxα=Φ˜(x).(32)

Hence, in terms of the original variable x, the Fréchet extreme value distribution becomes

ϕ˜(x)=dΦ˜(x)dx=Nαxα1eNxα.(33)

Fréchet: An Example

We consider the distribution

p(x)={0forx1,αxα1forx>1,(34)

The corresponding cumulative probability is given by

P(x)={0forx1,1xαforx>1.(35)

Using Eq. 33, we find the corresponding Fréchet extreme value distribution to be

ϕ˜(x)=Nαxα1eNxα,(36)

valid for all x>1. We show p(x) and the corresponding ϕ˜(x) for α=3 and N=100 and N=1,000 in Figure 2A.

FIGURE 2
www.frontiersin.org

FIGURE 2. (A) The curve that has its maximum at x=1 is the probability distribution 34 with α=3. The curve that has its maximum in the middle is ϕ˜(x), Eq. 36 with N=100 and the curve that has its maximum to the right is ϕ˜(x) with N=1,000. (B) The histograms shown here are based on data according to the probability distribution 34 with α=3. The histogram having its maximum to the left shows all the generated data. The histogram having its maximum in the middle shows the largest number among each sequence of numbers of length 100, and the histogram having its maximum to the right shows the largest number among each sequence of numbers of length 1,000. For each sequence length, 107 such sequences were generated.

In order to compare with numerical results, we generate numbers distributed according to 34 by solving the equation P(x)=r where r is drawn from a uniform distribution on the unit interval. From Eq. 35, we get

x=r1/α.(37)

We generate a sequence of numbers using this algorithm, grouping them together in sequences of N=100 or N=1,000. We generate 107 such sequences. The histograms based on the random numbers themselves generated with Eq. 37, and of the extreme values for each sequence of length either 100 or 1,000 we show in Figure 2B. This figure should be compared to Figure 2A.

Gumbel Class

We now assume we have a probability distribution that takes the form

p(x)=f(x)ef(x)forx>x0,(38)

where f(x)=df(x)/dx. We have that x0 is any number, positive or negative, and f(x) is an increasing function of x. We will later on introduce a sufficient criterion imposed on p(x) to produce the Gumbel distribution, see Eq. 53. This criterion is equivalent to f(x) fulfilling

limxddx(1f(x))=0.(39)

This criterion is e.g., fulfilled by any polynomial f(x).

The cumulative probability is

P(x)=1ef(x)forx>x0.(40)

We do not care about the form of p(x) or P(x) for xx0.

The extreme value cumulative probability for N samplings is given by

P(x)N=[1ef(x)]N,(41)

for x>x0. We introduce the variable change

u˜=f(x)f(xN),(42)

where xN is given by

P(xN)=11N.(43)

Even though xN is defined by 43, we may interpret its meaning. We do so in the conclusion, see Eq. 71. From Eq. 40 we then have that

f(xN)=lnN.(44)

Let us now define

Δx=xxN.(45)

We then expand f(x) around xN,

f(x)=f(xN+Δx)=n=0f(n)(xN)n!Δxn,(46)

where f(n)(x)=dnf(x)/dxn. If we now set

Δx=1f(xN),(47)

so that the first order term in the expansion becomes constant as N increases, we will have that

f(xN)Δx+n=2f(n)(xN)n!Δxn=1+n=2f(n)(xN)n!f(xN)n.(48)

Hence, if we have that

limNf(n)(xN)f(xN)n=0,(49)

for n2, then in this limit, we will find

f(x)=f(xN)+f(xN)Δx=f(xN)+u,(50)

where we define

u=f(xN)Δx=Np(xN)(xxN).(51)

Here we have used Eqs (40) and (44).

Sufficient Criterion for the Gumbel Class

If we combine Eq. 49 for n=2 with Eqs 38 and 40, we find

limNf(xN)f(xN)2=limNddx[1P(x)p(x)]x=xN=0,(52)

which is equivalent to

limxddx[1P(x)p(x)]=0.(53)

Equation 53, which is equivalent to Eq. 39, is in fact a sufficient condition for 49 to hold for all n>1. We may show this through induction. We have that

f(n+1)(x)f(x)n+1=1f(x)ddx(f(n)(x)f(x)n)+f(n)(x)f(x)n+2.(54)

If condition 52 is fulfilled, that is when the expression above is zero in the limit x for n=2, we also have that

limNf(3)(x)f(x)3=0,(55)

since both terms on the right hand side of Eq. 54 are zero in this limit. We now assume Eq. 49 to be true for some n>3. We then have that

limNf(n+1)(xN)f(xN)n+1=0,(56)

again due to both terms on the right hand side of Eq. 54 are zero in this limit. This completes the proof.

Return to the Derivation

We now combine Eq. 42 with Eq. 41 to find

P(x)N=[1euf(xN)]N=[1eulnN]N=[1euN]N.(57)

In the limit of N, this becomes

Φ(u)=limNP(x)N=eeu,(58)

which is the Gumbel cumulative probability. Here <u<. The Gumbel probability density is given by

ϕ(u)=dΦ(u)du=eueu.(59)

We express the Gumbel cumulative probability in terms of the original variable x using Eq. 51,

Φ(u)=Φ(Np(xN)(xxN))=eeNp(xN)(xxN)=Φ˜(x).(60)

Hence, in terms of the original variable x, the Gumbel extreme value distribution becomes

ϕ˜(x)=dΦ˜(x)dx=Np(xN)eNp(xN)(xxN)eNp(xN)(xxN).(61)

An Example: The Gaussian

Here is an example: the Gaussian. The Gaussian probability density is given by

p(x)=ex2/2σ2πσ,(62)

where σ is the square of the standard deviation. The cumulative probability is

P(x)=12[1+erf(x2σ)],(63)

where erf(z) is the error function. In order to verify that the Gaussian generates the Gumbel extreme distribution, we use the sufficient condition 53,

limxddx[1P(x)p(x)]=limxπ2σex2/2σx[1erf(x2σ)]=0.(64)

The Gaussian cumulative probability in Eq. 63 has the asymptotic form

P(x)=1σ2πex2/2σx,(65)

for large x. We determine xN solving Eq. 43 using this asymptotic form. We find

xN=σW(N22π),(66)

where W(z) is the Lambert W function, also known as the product logarithm, which is the solution to the equation W(z)exp[W(z)]=z. For large arguments, it approaches the natural logarithm, W(z)log(z) as z [16]. This gives us

Np(xN)=1σW(N22π),(67)

when inserting the expression for x=xN, Eq. 66 into Eq. 62. Thus we may now express the variable u in the Gumbel cumulative probability 57 in terms of the variables x, σ and N using Eq. 51,

u=x1σW(N22π)W(N22π).(68)

We show in Figure 3A the Gaussian and the corresponding Gumbel distributions for σ=1 and N=100 and N=1,000. We find that x100=2.375 and x1000=3.115. These are the confidence intervals for 99% and 99.9%.

FIGURE 3
www.frontiersin.org

FIGURE 3. (A) The Gaussian and the corresponding Gumbel distributions for σ=1 and N=100 and N=1,000. (B) The histograms shown here are based on data generated using the Box-Müller algorithm which produces numbers distributed according to a Gaussian. Here σ=1. The histogram with the maximum to the left shows all the generated data. The histogram with its maximum in the middle shows the largest number among each sequence of numbers of length 100, and the histogram with the rightmost maximum shows the largest number among each sequence of numbers of length 1,000. For each sequence length, 107 such sequences were generated.

We show in Figure 3B a histogram based on numbers distributed according to a Gaussian distribution using the Box-Müller algorithm [14]. These numbers were grouped together in sets of either N=100 or N=1,000 elements. I generated 107 such sets. The figure displays the two extreme distributions for the two set sizes. This figure should be compared to Figure 3A. In contrast to the two other extreme value distributions, we see that there are visible discrepancies between the calculated Gumbel distributions in Figure 3A and the extreme value histograms in Figure 3B. We see furthermore that the histogram for N=1,000 is closer to the calculated Gumbel distribution than the histogram for N=100. This is due to the very slow convergence induced by the Lambert W functions. Slow convergence is typical for the Gumbel extreme value distributions. This slow convergence has been analyzed and recently and through clever use of scaling methods remedied [17].

Concluding Remarks

We summarize the main results presented in this mini-review in Table I.

We have only discussed the distributions associated with the largest values of x except for the Weibull extreme value distribution, Section 2. It is, however, easy to work out: just transformxx. Otherwise, the story presented here is rather complete.

There is one remark that needs to be made, though. In the derivation of the Gumbel extreme value distribution, Section 4, we defined a variable xN in Eq. 43. First of all, xN defined in Eq. 43 may be calculated for any cumulative probability P(x) and it has an interpretation making it very useful.

The probability density for the largest among N numbers drawn using the probability distribution p(x) is given by

pN(x)=dP(x)Ndx=NP(x)N1p(x).(69)

We calculate the average of the cumulative probability P(x) for the extreme value based on N samples,

P(x)=P(x)pN(x)dx=01PNdP=NN+1=11N+1.(70)

For large N, we may write this as

P(x)=P(xN)=11N,(71)

using here Eq. 43. Hence, we may interpret xN as the x value corresponding to the average confidence interval of the largest observed value in sequences of N numbers. It is essentially the typical size of the extreme value for a sample of size N.

Author Contributions

The author confirms being the sole contributor of this work and has approved it for publication.

Funding

This work was partly supported by the Research Council of Norway through its Centers of Excellence funding scheme, project number 262644.

Conflict of Interest

The author declares that the research was conducted in the absence of any commercial or financial relationships that could be constructed as a potential conflict of interest.

Acknowledgments

I thank Eivind Bering, Astrid de Wijn, H. George, E. Hentschel, Srutarshi Pradhan, and Itamar Procaccia for numerous interesting discussions on this topic.

Footnotes

1

IID variables. Independent and identically distributed random variables, a terminology used in some communities.

REFERENCES

1. Gumbel EJ. Statistics of extremes. New York: Columbia University Press(1958).

Google Scholar

2. David HA. Order statistics. 2nd ed. New York: Wiley (1981).

Google Scholar

3. Galambos J. The asymptotic theory of extreme order statistics. Malabar, FL: Krieger(1987).

Google Scholar

4. Embrechts P, Klüppelberg C, Mikosh T. Modeling extreme events for insurance and finance. Berlin: Springer(1997).

Google Scholar

5. Coles S. An introduction to statistical modeling of extreme events. Berlin: Springer(2001).

Google Scholar

6. Bouchaud J-P, Mézard M. Universality classes for extreme-value statistics. J Phys Math Gen(1997). 30:7997. doi:10.1088/0305-4470/30/23/004

CrossRef Full Text | Google Scholar

7. Antal T, Droz M, Györgyi G, Rácz Z. 1/f noise and extreme value statistics. Phys Rev Lett(2001). 87:240601. doi:10.1103/physrevlett.87.240601

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Randoux S, Suret P. Experimental evidence of extreme value statistics in Raman fiber lasers. Opt Lett(2012). 37:500. doi:10.1364/OL.37.000500

CrossRef Full Text | Google Scholar

9. Taloni A, Vodret M, Costantini G, Zapperi S. Size effects on the fracture of microscale and nanoscale materials. Nat Rev Mater(2018). 3:211–24. doi:10.1038/s41578-018-0029-4

CrossRef Full Text | Google Scholar

10. Hansen A, Hemmer PC, Pradhan S. The fiber bundle model. Berlin: Wiley VCH(2015).

Google Scholar

11. Pal A, Eliazar I, Reuveni S. First passage under restart with branching. Phys Rev Lett(2019). 122:020602. doi:10.1103/PhysRevLett.122.020602

CrossRef Full Text | Google Scholar

12. Fortin J-Y, Clusel M. Applications of extreme value statistics in physics. J Phys Math Theor(2015). 48:183001. doi:10.1088/1751-8113/48/18/183001

CrossRef Full Text | Google Scholar

13. Majumdar SN, Pal A, Schehr G. Extreme value statistics of correlated random variables: a pedagogical review. Phys Rep (2020) 840:1. doi:10.1016/j.physrep.2019.10.005

CrossRef Full Text | Google Scholar

14. Press WH, Teukolsky SA, Vetterling WT, Flannery BP. Numerical recipes. 3rd ed. Cambridge: Cambridge University Press(2007).

Google Scholar

15. Rinne H. The Weibull distribution. Boca Raton: CRC Press (2008).

Google Scholar

16. Corless RM, Gonnet GH, Hare DEG, Jeffrey DJ, Knuth DE. On the LambertW function. Adv Comput Math (1996) 5:329–59. doi:10.1007/BF02124750

CrossRef Full Text | Google Scholar

17. Zarfaty L, Barkai E, Kessler DA. Accurately approximating extreme value statistics(2020). arXiv:2006.13677.

Google Scholar

Keywords: extreme value statistics, statistical analysis, Weibull analysis, Gumbel distribution, Frechet distribution, Weibull distribution

Citation: Hansen A (2020) The Three Extreme Value Distributions: An Introductory Review. Front. Phys. 8:604053. doi: 10.3389/fphy.2020.604053

Received: 08 September 2020; Accepted: 22 October 2020;
Published: 10 December 2020.

Edited by:

Matjaž Perc, University of Maribor, Slovenia

Reviewed by:

Arnab Pal, Tel Aviv University, Israel
Haroldo V. Ribeiro, State University of Maringá, Brazil

Copyright © 2020 Hansen. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Alex Hansen, Alex.Hansen@ntnu.no