Approximating the three parameters of Gumbel model through Lindley Bayes

Introduction

Bayes estimators are used to estimate the parameter of a Gumbel distribution under a Bayesian framework. For the three parameter Gumbel distribution (α, ꞵ, γ). The Bayesian estimation provides a way to incorporate prior knowledge or beliefs about the parameters in form of distribution in the estimation process. Bayesian estimation allows the use of prior knowledge or expert opinion about the parameter (α, ꞵ, γ) through prior distributions. In cases where sample size is small or data is spares, frequentist methods (e.g. maximum likelihood estimation) may yield unstable estimates. Bayesian methods improve the estimation process by taking advantage of prior information. Bayesian estimators provide posterior distribution for the parameters, provide not just point estimates but also posterior intervals, which quantify uncertainty about the estimates. For distributions like the three parameter Gumbel, where maximum likelihood estimation might be challenging or computationally intense. Bayes estimator minimizes the expected posterior loss, making them adaptable to different loss function (e.g. squared error loss). This makes the estimation more optimized for particular situation. Extreme value distributions have been an important topic of research in applied probability and statistics. Kotz and Nadarajah (2000) [1] presented a detailed theoretical framework for extreme value distributions and their applications, whereas Lawless (2003) [2] emphasized statistical models for lifetime data, which are closely related to reliability and survival analysis. From a Bayesian perspective, Bernardo and Smith (2000) [3] and Robert (2007) [4] offered fundamental contributions by developing the theoretical and computational foundations of Bayesian inference. In the same direction, Lindley (1980) [5] introduced an approximation method for Bayesian analysis, which provides an effective solution for cases where posterior integrals are intractable.

Later, Tierney and Kadane (1986) [6] extended this work by proposing accurate approximations for posterior moments and marginal densities. These works provided the basis for developing Approximate Bayes Estimation methods. In the field of reliability analysis, Sinha (1986) [7] discussed life testing approaches that further motivate the use of flexible lifetime distributions. Within this framework, several authors have studied Gumbel and generalized Gumbel distributions. Raqab and Kundu (2005) [10] discussed Bayesian estimation for the generalized Gumbel distribution, while Sharma and Kumar (2013) specifically applied Lindley's approximation for Bayesian inference in generalized Gumbel models. Kumar and Lalhita (2012) also explored Bayesian estimation in the Gumbel distribution under various loss functions. More recently, Okumu et al. (2024) [9] extended the study of the three-parameter Gumbel distribution by proposing formulations and estimation procedures. Similarly, Oluwafunmilola et al. (2023) [8] introduced the Lindley Exponentiated Gumbel distribution and demonstrated its applicability to environmental data, highlighting the continuing relevance of Lindley's approximation in modern distribution theory. In parallel, Srivastava and Yadav (2018) [13] made contributions to the Bayesian estimation of the generalized compound Rayleigh distribution, applying approximate Bayes techniques under different loss functions. Their later works (Yadav & Srivastava, 2018) [13] considered estimation under entropy and precautionary loss functions, further showing the versatility of Lindley's approximation in lifetime distributions. Overall, earlier studies show that Bayesian estimation and Lindley's approximation have been applied to generalized Gumbel, compound Rayleigh, and related distributions. However, there is still room to study the three-parameter Gumbel distribution more closely. In particular, most past research has focused on generalized versions or on different loss functions, while only a few works have directly looked at the three-parameter case under squared error loss using Lindley's method. This study addresses that gap by proposing Approximate Bayes Estimators (ABEs) for the parameters (α, β, γ) of the three-parameter Gumbel distribution and comparing them with the traditional Maximum Likelihood Estimation (MLE) through simulation.

The Gumbel Distribution

The "Gumbel Distribution" is named after Emil Julius Gumbel (1891-1966), a German Mathematician and Statistician. He is considered the founder of the Gumbel distribution, which he introduced as part of his work on the extreme value theory. He developed the distribution to model the distribution of the maximum (or minimum) values of large datasets. "The transition from a two parameter Gumbel distribution to three parameter Gumbel distribution involves adding a shape or shift (location). It is useful when the data shows skewness, asymmetry, or systematic shift that cannot be explained by just two parameters".

1. Let x be a random variable representing the data points or observations. In practical applications, it could represent extreme values such as maximum temperatures, flood levels, or financial losses. f(x) be the probability density function value at x.

Where, γ is the location parameter determines the location of the distribution. ꞵ > 0 is the scale parameter controls the spread of the distribution. α is the shape parameter. For x ≤ γ the pdf is 0 because the Gumbel distribution is not defined below this threshold when using a shift form. For x > γ the pdf follows the exponential term.

Simplify the likelihood function

Differentiating equation number (6.3) with respect to (α,ꞵ,γ) yield respectively we get

Setting the derivatives ion equation (3.4), (3.5) and (3.6) to zero and solving for (α,ꞵ,γ) gives the maximum likelihood estimators (MLEs) of the parameters: α ̂MLE , β ̂MLE and γ ̂MLE . These estimates can be obtained using the Newton-Raphson method.

 Bayes estimators of ꞵ with known parameter α and γ under Linex loss function (LLF)

If 𝛂 ̂ and 𝛄 ̂ is known we assume (a, b) as conjugate prior for ꞵ as:-

Combining likelihood function (3.2) and prior density (4.1) we obtain the posterior density of ꞵ in the form of

Then we combine the likelihood and the prior we obtain the posterior after solving the previous equation, we get the posterior

Bayes estimate under squared error loss function (Self)

Β ̂BS is the posterior mean given by-

Bayes estimators with unknown α,ꞵ and γ

Joint prior density α,ꞵ and γ is given by:-

Joint posterior with likelihood equation number (3.2) and (6.4) we get;

The approximate Bayes estimators

To apply Lindley approximation in the equation number (7.1)

Likelihood function from equation number (3.2) L(x) = β n e (-β(x-γ-α)) e -∑ e (-β(x-γ-α)) (x;α,ꞵ,γ >0)

Log likelihood of above equation

∂γ 2 = -ꞵ 2 e -β(x-γ-α) (7.15)

= n-e -β(x-γ-α) +ꞵ(x -γ -α). e (-β(x-γ-α)) (7.16)

= n-e -β(x-γ-α) +ꞵ(x -γ -α). e (-β(x-γ-α)) (7.17 ] = ꞵ 2 (x -γ -α)e -β(x-γ-α) (7.30)

∂γ ∂α ] = -ꞵ 3 e -β(x-γ-α) (7.31) ] = e -β(x-γ-α) (-2β + β 2 (x -γ -α)) (7.37)

∂ꞵ 2 ] = -2(x -γ -α)e (-β(x-γ-α)) + β(x -γ -α) 2 e (-β(x-γ-α)) (7.38)

∂γ ∂ꞵ ] = ꞵ 2 (x -γ -α)e -β(x-γ-α) (7.39)

∂γ 2 ] = -ꞵ 3 e -β(x-γ-α) (7.40) = σ 11 ꞵ 3 e -β(x-γ-α) + 2σ 12 ꞵ 2 (x -γ -α)e -β(x-γ-α) + 2σ 13 (-ꞵ 3 e -β(x-γ-α) ) + 2σ 23 e -β(x-γ-α) (-2β + β 2 (x -γ -α))

+σ 22 2(x -γ -α)e -β(x-γ-α) -β(x -γ -α) 2 e (-β(x-γ-α)) +σ 33 ꞵ 3 e -β(x-γ-α) (8. = σ 11 ꞵ 3 e -β(x-γ-α) + 2σ 12 ꞵ 2 (x -γ -α)e -β(x-γ-α) + 2σ 13 (-ꞵ 3 e -β(x-γ-α) ) + 2σ 23 e -β(x-γ-α) (-2β + β 2 (x -γ -α)) +σ 22 2(x -γ -α)e -β(x-γ-α) -β(x -γ -α) 2 e (-β(x-γ-α)) +σ 33 ꞵ 3 e -β(x-γ-α) (8.7)

Were, ϕ 1 = (v 1 a 1 + v 2 a 2 + v 3 a 3 + a 4 + a 5 ) (8.9)

Approximate Bayes Estimate

The above equation (9.4) is evaluated by method of Lindley Approximation, whose simplified form is equation number (9.1) replace θ by V αβγ in equation number (9.4) and (9.1).

Special cases:-

Analysis of Alpha Estimates

The first chart displays the estimation behaviour of the parameter α under both the MLE and ABS methods. At smaller sample sizes (N=20, 30), both estimators show some variation, particularly at smaller sample sizes. However, the Approximate Bayesian Estimator (ABS) generally provides greater stability and produces estimates that are closer to the true parameter values than those from the Maximum Likelihood Estimator (MLE). As the sample size increases beyond 40, both estimators progressively become more reliable with their estimates approaching the actual parameter values indicating that both are consistent.

Analysis of Beta Estimates

The second chart presents the estimation results for the parameter β. Here, both the MLE and ABS estimators perform very well across all sample sizes. While small fluctuations are observed at lower sample sizes, these are less significant than the α estimates. Both estimators converge rapidly to the true parameter value as the sample size increases, indicating high efficiency and consistency. However, the ABS estimator continues to show a marginal improvement in stability for smaller sample sizes, highlighting its reliability in real world situations with limited data.

Analysis of Gamma Estimates

The third chart displays the behaviour of the gamma (γ) estimates obtained through both the Maximum Likelihood Estimator (MLE) and the Approximate Bayesian Estimator (ABS) across increasing sample sizes. A clear characteristic of the plot is the decreasing trend in the estimates with increasing sample size, which is largely due to the increased sensitivity in estimating the location parameter (γ) at smaller sample sizes. The γ parameter represents the minimum or threshold value in the distribution. In small samples, there is a higher likelihood that the true minimum value is not captured due to limited data, which causes the estimator to overestimate γ. As a result, for smaller sample sizes, both MLE and ABS initially produce gamma estimates that are higher than the true parameter value, showing a positive bias.

As the sample size increases, the probability of observing values closer to the true minimum rises. This additional information allows both estimators to correct the earlier overestimation, causing the gamma estimates to gradually decrease and converge towards the true value. The Approximate Bayesian Estimator shows a smoother and more consistent decrease than MLE, mainly because prior knowledge in the Bayesian method helps stabilize estimates when data is limited. Thus, the downward trend in the gamma estimates reflects the correction of small-sample bias in location parameter estimation as more information becomes available with increasing sample sizes. This behaviour is well-known in reliability analysis and extreme value modelling, where location parameters are highly sensitive to sample extremes.

Conclusion

The study shows that both Maximum Likelihood Estimator (MLE) and Approximate Bayesian Estimator (ABS) perform well in estimating the parameters α, β, and γ of the three-parameter distribution, especially as the sample size increases. However, the ABS estimator generally provides more stable and accurate results for smaller sample sizes due to its ability to include prior information. For α, ABS reduces variability and bias better than MLE in small samples. For β, both estimators perform almost equally well, with ABS having a slight advantage in stability. For γ, both estimators initially overestimate due to the small sample size, but ABS adjusts more smoothly as more data becomes available. "Overall, the Approximate Bayesian Estimator performs better with smaller sample sizes, whereas both methods yield comparable outcomes for larger samples, indicating stable and accurate estimation performance.

References

1. Kotz S, Nadarajah S. Extreme value distributions: theory and applications. Singapore: World Scientific; 2000.
2. Lawless JF. Statistical models and methods for lifetime data. 2 nd Ed., Hoboken, NJ: Wiley; 2003.
3. Bernardo JM, Smith AFM. Bayesian theory. Chichester: Wiley; 2000.
4. Robert CP. The Bayesian choice: from decision-theoretic foundations to computational implementation. 2 nd Ed., New York: Springer; 2007.
5. Lindley DV. Approximate Bayesian methods. Trab Estad Investig Oper. 1980;31(1):223-237.
6. Tierney L, Kadane JB. Accurate approximations for posterior moments and marginal densities. J Am Stat Assoc. 1986;81(393):82-86.
7. Sinha SK. Reliability and life testing. New Delhi: Wiley Eastern; 1986.
8. Oluwafunmilola AO, Opeyimika OO, Sunday OM. A new approach of presenting Lindley exponentiated Gumbel distribution with application to environmental data. Int J Membr Sci Technol. 2023;10(2):2649-2657.
9. Okumu KK, Ouno JO, Karanjah AN. Three-parameters Gumbel distribution: formulation and parameter estimation. Int J Stat Distrib Appl. 2024;10(2).
10. Raqab MZ, Kundu D. Bayesian estimation for the generalized Gumbel distribution. J Stat Comput Simul. 2005;75(10):787803.
11. Sharma VK, Kumar A. Bayesian estimation in generalized Gumbel distribution using Lindley's approximation. Int J Stat Syst. 2013;8(2):139-149.
12. Kumar D, Lalhita S. Bayesian estimation in Gumbel distribution under different loss functions. Int J Agric Stat Sci. 2012;8(1):65-71.
13. Srivastava U, Yadav P. Approximate Bayes estimate for the parameters of generalized compound Rayleigh distribution under precautionary loss function. Int J Creat Res Thoughts. 2018;6(2).