Innovative approaches for modifying probability distributions

Introduction

Distribution fitting is essential in statistical modelling, underpinning key practices like inference, prediction, and evidence-based decision-making. A probability distribution provides a theoretical framework to describe the frequency and variability of potential outcomes. During the fitting process, various theoretical distributions are evaluated against the data, with iterative adjustments made to the parameters based on goodness-of-fit measures. The outcomes include ranked distributions, optimized parameter estimates, and an assessment of their alignment with the data. Identifying a statistical distribution that best represents a given dataset, known as distribution fitting, is a critical process in data analysis. This involves estimating the parameters of the selected distribution to closely match the observed data. The choice of a suitable distribution is influenced by the characteristics of the dataset and the goals of the analysis. Advanced methods such as the Method of Moments, Maximum Likelihood Estimation (MLE), and Bayesian Estimation are often used to refine parameter estimates and minimize discrepancies between the model and the data. Probability distributions are essential in analyzing real-world data within the field of Biostatistics. Biostatistics, a field merging statistical methodologies with biological sciences, plays a pivotal role in advancing public health, clinical research, genetics, and epidemiology. It emphasizes rigorous study design, data analysis, and interpretation based on statistical principles, addressing complex biological and medical challenges with quantitative precision. In response to the limitations of traditional models for analyzing intricate data, numerous oneparameter lifetime distributions have been introduced. These models address the variability and heterogeneity within populations that classical approaches often fail to capture. Finite mixture models have proven particularly valuable, offering a flexible framework for representing data as combinations of multiple probability distributions. Widely applied in fields such as reliability analysis, survival studies, and risk modelling, these models include mixtures of Exponential and Gamma distributions, which consistently outperform their traditional counterparts.

Marshall-Olkin Method

The Marshall-Olkin [1] distribution provides a systematic approach to introducing additional parameters into a probability distribution, enabling greater flexibility in modelling hazard rates. This technique is widely utilized in fields such as survival analysis, reliability theory, and risk assessment. The Marshall-Olkin method modifies a baseline cumulative distribution function (CDF), F(x), by incorporating an additional parameter α to adjust the hazard rate. This transformation results in a new, more versatile distribution with properties tailored to specific analytical needs. Transformation: Given a baseline distribution with CDF F(x), the Marshall-Olkin transformation is defined as:

,0 ≤ α ≤1, α is the Shape parameter introduced to control the hazard rate: A parameter controlling the failure rate. This is used in reliability modelling to introduce proportional hazards, in fitting biomedical data, fitting skewed distributions, etc. Impact of Parameter α: i) When α=1, The transformation reduces to the baseline distribution F(x). ii) When α>1, The modified distribution exhibits an increased hazard rate. iii) When 0<α<1, The modified distribution displays a decreased hazard rate. Applications: i) Survival Analysis: By adjusting the hazard rate, the Marshall-Olkin method models survival times under varying conditions, including systems with changing failure rates. ii) Reliability Theory: The flexibility of the transformed distribution makes it suitable for modelling the lifetime of complex systems, especially those with nonconstant failure rates. iii) Risk Assessment: It allows for better representation of risk in financial, environmental, and engineering domains by tailoring the tail behaviour of the distribution. Advantages: i) Introduces flexibility without requiring entirely new distribution forms. ii) Enables finetuning of hazard rates to fit empirical data more closely. iii) Maintains compatibility with existing probabilistic frameworks. The Marshall-Olkin method is a powerful and versatile tool for enhancing the applicability of traditional probability distributions, making it indispensable in domains where hazard rate variation is a critical factor. Sandhya and Prasanth [2] derived Marshall-Olkin discrete Uniform (MODU) distribution, and the goodness of the distribution is tested with real data. Jose and Alice [3] studied about the Marshall-Olkin family of distributions, and its applications in time series modelling and reliability. Jose and Krishna [4] detailed the Marshall-Olkin extended uniform distribution. Sandhya and Prasanth [5] analysed a generalized geometric distribution and its application

Harris Method

Proceeding as the MO set up mentioned, let F(x) be the CDF of a discrete random variable X, f(x) be its PMF and let F(x) be the Survival Function(SF) of a distribution, then we can write the SF of a new distribution by Harris Method [6] and we get a new Harris family of SF by adding a new parameter θ as, H¯(x, θ, k) = θF¯(x) 𝑘 (1-(1-θ)F¯(x) 𝑘 , θ > 0, k ε N. Prasanth and Sandhya [7] , detailed a Generalized Discrete Harris Uniform Distribution. Sandhya and Prasanth [8] explained the Harris Geometric Distribution and applications. Sandhya et al. [9] discussed about the Harris family of Discrete Distributions and characterisation. Also, Sandhya et al. [10] explained the Characterizations of the extended geometric, Harris, negative binomial, and gamma distributions.

Mixed Distributions

Mixed distributions combine two or more probability distributions to create a single model that captures more complex data patterns, including asymmetry and multimodality. This approach is particularly useful for datasets that exhibit heterogeneous or multimodal characteristics. A mixture of n distributions f 1 (x),f 2 (x),…,f n (x) with weights p 1 ,p 2 ,…,p n (where ∑p i = 1) then f(x) = ∑p i f i (x). Hence to combine two distributions: Define a mixture distribution: f(x) = p⋅f 1 (x)+(1-p) ⋅f 2 (x) where f 1 (x) and f 2 (x) are PDFs of two distributions, and p∈ [0,1]. Generally used in Modelling bimodal distributions (e.g., heights of men and women in a population) and in clustering algorithms like Gaussian Mixture Models (GMMs). The Properties of these models are: i) Asymmetry: The resulting distribution can exhibit asymmetry, depending on the characteristics and weights of the component distributions. ii) Multimodality: Mixed distributions can capture multimodal patterns, where the data has multiple peaks or clusters. Main Applications are: i) Modelling Bimodal Distributions: For example, a mixed distribution might model the heights of men and women in a population, where each subgroup follows a distinct normal distribution. ii) Clustering Algorithms: Mixture distributions are foundational in clustering techniques like Gaussian Mixture Models (GMMs), which classify data into multiple groups based on the underlying distribution patterns. iii) Risk Assessment: Mixed models are often used to capture the behaviour of data with extreme values or distinct subpopulations, such as insurance claims or stock returns. The Advantages of the model are: i) Flexibility: By combining component distributions, mixed models can adapt to complex data structures. ii) Realism: They better represent heterogeneous or non-uniform datasets compared to singledistribution models. iii) Interpretability: Each component distribution can be associated with a specific subgroup or characteristic of the data. Mixed distributions provide a robust framework for modelling complex and diverse datasets, making them indispensable in statistical analysis and machine learning. Feller [11] explores mixed distributions within the broader framework of probability theory, focusing on their construction and characteristics. Johnson et al.. [12] provide an in-depth analysis of mixed distributions, specifically in the context of continuous distributions. More recently, Al-Omari [13] examined the mixture of Shanker and Gamma distributions, highlighting its applications in engineering data.

Weighted Distributions Method

Weighted or Biased Distributions: A weighted distribution is a method of modifying a baseline distribution by introducing a weight function w(x), which emphasizes or deemphasizes certain regions, it may create asymmetry. Modification: Weight the PDF f(x) with a function w(x):

For example: i) Let w(x)=e kx , to skew the distribution and forms Exponentially weighted distributions. ii) Let w(x)=x k: for weighting by a power of x. The main applications are at i) Modelling biased samples in surveys iii) Adjusting likelihoods in Bayesian analysis. iii) Fitting asymmetric data Here are a few straightforward examples of weighted distributions: i) Survival Analysis: In studies where patients are selected based on the severity of their condition, the observed survival times may deviate from the true distribution of survival times across the general population. ii) Ecology: Surveys that record animals tend to favor larger or more visible species, resulting in a sample that not accurately reflect the actual distribution of all animals in the environment. iii) Reliability Engineering: When defective products are more prone to early failure, the failure times recorded may not be representative of the overall product population.

Length Biased Distributions: For a random variable X with PDF f(x), then the area-biased PDF is:

where E[X] is the expected value of X. Here a special case of the weighted distribution when the weight function w(x) = x itself. it specifically applies to time-based phenomena, such as durations or waiting times.

Area Biased Distributions: Occur when the probability of selecting an observation is proportional to its square. This happens in cases where larger values are more likely to be sampled. For a random variable X with PDF f(x), then the area-biased PDF is:

where E[X] is the expected value of X. Here a special case of the weighted distribution when the weight function w(x) = x 2 . Rao and Pandey [14] , described the Parameter estimation of area biased Ailamu-jia distribution. Fazal [15] studied about the Area-biased Poisson exponential distribution with applications. Chouia et al.. [16] , explained a new size biased distribution with application. Nwikpe [17] expressed the Twoparameter Nwikpe (TPAN) distribution with application.

Roshni et al.. [18] , explained a generalized area-biased Powerishita distribution, properties and applications.

Decreasing Uniform Scale (DUS) Transformation

The Decreasing Uniform Scale (DUS) transformation is a statistical technique applied to random variables, particularly valuable in reliability analysis and survival modelling. This method is instrumental in characterizing distributions with decreasing failure rates (DFR) and modifying baseline distributions to exhibit decreasing hazard functions. Generally, the DUS transformation is employed to adjust a baseline distribution's properties, specifically its tail behaviour. For a given baseline distribution F(x), the DUS transformation generates a modified distribution:

, k is a parameter controlling the shape of the transformation. As, k→∞, F * (x)→1 resulting in heavy-tailed behaviour. And when k=1, F * (x) equals F(x), preserving the original distribution.

Let F(x) is the CDF of the baseline distribution, then the CDF of the new DUS transformed distribution is as follows:

Then the PDF, g(x) = [ 𝑒 𝐹(𝑥) 𝑓(𝑥)] where f(x) is the PDF of the initial (baseline) distribution. Applications: i) Modelling extreme values or heavy tails in finance and insurance, ii) Tailoring distributions for risk analysis. Anakha and Chacko [19] detailed the DUS-Kumaraswamy distribution as a Bathtub shaped failure rate model. Gauthami and Chacko [20] described the DUS transformation of inverse Weibull distribution as an upside-down failure rate model. Thomas and Chacko [21] observed the Power generalized DUS transformation of exponential distribution. Also, Thomas and Chacko [22] detailed the Power generalized DUS transformation in Weibull and Lomax distributions.

Simulation analysis

Simulation analysis is a versatile tool in probability and statistics, widely employed to explore the behaviour of probability distributions. By generating random samples from specified distributions, it enables researchers to analyze data, uncover statistical properties, evaluate estimation techniques, and assess real-world applications. Key benefits of simulation analysis include: Exploring Theoretical Properties: Gaining deeper insights into the behaviour and characteristics of distributions. Estimating Moments and Measures: Computing key statistical metrics such as mean, variance, skewness, and kurtosis. Model Comparison: Evaluating the performance of different probability models. Assessing Estimation Techniques: Testing and refining parameter estimation methods. Conducting Hypothesis Testing: Performing statistical tests to validate hypotheses. Goodness-of-Fit Analysis: Determining how well a distribution aligns with observed data. Simulation provides a practical framework for generating data, visualizing outcomes, estimating parameters, and validating models, making it an invaluable approach for studying probability distributions and addressing complex statistical challenges.

Type of Non-Symmetric Distributions 4.1 Non-Symmetric Distributions

Non-symmetric distributions are characterized by their asymmetry, where one side of the distribution deviates from the other. These distributions are highly applicable in modelling real-world data with inherent skewness or asymmetry. Below are some types and transformations that yield non-symmetric distributions:

Skewed Distributions

A skewed distribution has a longer or fatter tail on one side of its peak, quantified by the skewness parameter. Common examples include: Log-normal Distribution, Gamma Distribution, Weibull Distribution and Exponential Distribution

Truncated or Censored Distributions:

These distributions are derived by restricting or censoring part of the range, creating asymmetry. For instance: A symmetric distribution X∼N (μ, σ 2 ) truncated to [a,b] where a≠ -b, becomes non-symmetric.

Generalized Distributions:

1) Skew-Normal Distribution: Adds a skewness parameter α to the normal distribution. Probability density function, (PDF): f(x;ξ,ω,α)=

) where ϕ is the standard normal PDF and Φ is the CDF. 2) Generalized Gamma Distribution: Incorporates additional shape and scale parameters to control skewness and tail behaviour, providing more flexibility.

Transformations to Non-Symmetric Forms:

Symmetric distributions can be modified to achieve asymmetry through transformations, such as: Y=X+ϵ, where ϵ is an asymmetric random noise term. Also, Y=log(X), for Uniform (a, b). These methods and distributions offer powerful tools for analyzing and modelling data with non-symmetric characteristics, accommodating the complexity of real-world phenomena.

Goodness of fit

Goodness of fit is a statistical metric used to assess the alignment between a probability distribution or statistical model and a given real-world dataset. It evaluates how effectively the chosen theoretical distribution represents the observed data. To ensure accurate modelling and predictions, goodness of fit typically involves:  Graphical Analysis: Visual tools such as histograms, Q-Q plots, or P-P plots are used to compare the theoretical distribution with the observed data.  Statistical Tests: Formal tests like the Kolmogorov-Smirnov test, Chi-Square test, or Anderson-Darling test provide quantitative measures of fit.  Information Criteria: Metrics like Akaike Information Criterion (AIC) or Bayesian Information Criterion (BIC) are employed to compare multiple models and select the most suitable one based on goodness of fit and model complexity.

By integrating these methods, researchers can systematically evaluate and identify the distribution that best represents their data, leading to more reliable modelling and predictive accuracy.

Probability distributions describe the behaviour of random variables, with their structural properties providing key insights that enable analysis and comparison. These properties include: Support: The range of values a random variable can assume, Expected Value: A measure of central tendency, representing the average outcome of the distribution. Variance: A measure of dispersion, indicating the spread or variability of the data. Skewness: An assessment of the asymmetry in the distribution's shape. Kurtosis: A measure of the tail heaviness or the peak sharpness of the distribution. Dependence Structure: Captures relationships and interactions between multiple variables within a distribution. A thorough understanding of these structural properties is fundamental for interpreting probability distributions, evaluating their characteristics, and applying them effectively to real-world data. This knowledge facilitates meaningful comparisons between distributions and supports informed decision-making in statistical analysis.

Model Selection Criteria

Selecting the optimal statistical model from a set of candidates requires careful evaluation of its performance based on multiple factors, including accuracy, complexity, interpretability, and generalizability. Below are the most commonly used criteria:

Akaike Information Criterion (AIC)

Balances goodness of fit with model complexity. Formula: AIC = 2k-2ln(L), where k is the number of parameters and L is the likelihood of the model. AIC does not give an absolute measure of model quality but rather a relative comparison between models. If two models have similar AIC values, the one with fewer parameters is preferred. It can handle complex model structures. A lower AIC value indicates a better tradeoff between accuracy and simplicity.

Bayesian Information Criterion (BIC)

Similar to AIC but penalizes complexity more heavily for larger datasets. Formula: BIC=k ln(n)-2 ln (L), where n is the sample size. BIC penalizes complexity more than AIC because it includes the term k (ln (n)). For small datasets, AIC and BIC often select the same model. For large datasets, BIC strongly favours simpler models. Lower BIC is better. If AIC selects a complex model but BIC selects a simpler one, you should prefer BIC if you prioritize simplicity. Preferred in Bayesian and probabilistic models. Also, when dealing with large datasets or when overfitting is a concern.

Corrected Akaike's Information Criterion (AICc)

The Corrected Akaike's Information Criterion (AICc) is an enhanced version of the AIC, specifically designed to address issues related to small sample sizes. While the standard AIC is effective for large datasets, it tends to favour overly complex models when applied to smaller datasets, increasing the risk of overfitting. To mitigate this, AICc introduces a correction factor to account for sample size. AICC = AIC + increases when n is small, penalizing models with more parameters. This ensures that AICc discourages overfitting in scenarios with limited data. Convergence to AIC for Large sample size n: i.e., As the sample size n→∞, the correction term becomes negligible, and AICc converges to AIC. Thus, AICc is equivalent to AIC for large datasets, retaining its effectiveness. AICc provides a robust tool for model selection in scenarios with limited data, ensuring that model complexity is appropriately penalized. For researchers working with small sample sizes, AICc helps strike a balance between goodness of fit and model parsimony, promoting more reliable and interpretable results. There are some more similar criterions and methodologies like, Deviance Information Criterion (DIC): Commonly used in Bayesian statistics. Combines model fit and complexity to evaluate hierarchical or complex models. A lower DIC value indicates a better model. Cross-Validation: Splits the dataset into training and testing sets to evaluate the model's predictive performance. Common approaches include k-fold crossvalidation and leave-one-out cross-validation. Prioritizes generalizability to unseen data. Adjusted R-Squared: Adjusts the R-squared value to account for the number of predictors in a regression model. Ensures that adding variables does not artificially inflate the fit. Likelihood Ratio Test: Compares the fit of two nested models (one a simpler version of the other). Determines if the additional complexity of the more elaborate model is justified. Schwarz Criterion (SC): Synonymous with BIC and emphasizes penalizing complexity in large datasets. Predictive Information Criteria (PIC): Focuses on the predictive accuracy of the model rather than its fit to the training data.

Conclusion

In conclusion, this study detailed innovative techniques for modifying standard probability distributions to better accommodate non-symmetrical real-world data and identify appropriate fits for such skewed datasets. Several wellestablished methods were explored for these transformations, including the Marshall-Olkin method, Harris method, weighted distribution method, mixed-type distribution method, and the Decreasing Uniform Scale (DUS) transformation method. Additionally, the effects of these methods were analyzed, and suitable methodologies for identifying the best fit, such as graphical analysis, information and statistical tests, were explained.

References

1. Marshall AW, Olkin I. A new method for adding a parameter to a family of distributions with application to the exponential and Weibull families. Biometrika. 1997;84(3):641-652. https://doi.org/10.1093/biomet/84.3.641
2. Sandhya E, Prasanth CB. Marshall-Olkin discrete uniform distribution. J Probab. 2014;2014:979312. https://doi.org/10.1155/2014/979312
3. Jose KK, Alice T. Marshall-Olkin family of distributions: Applications in time series modelling and reliability. J.C. Publications, 2005.
4. Jose KK, Krishna E. Marshall-Olkin extended uniform distribution. ProbStat Forum, 2011.
5. Sandhya E, Prasanth CB. A generalized geometric distribution. In: Proceedings of the International Conference on Frontiers of Statistics and Applications & 32 nd Annual Conference of Indian Society for Probability and Statistics. Pondicherry University, 2012, p. 261-9. ISBN: 978-93-82338-78-9.
6. Harris TE. Branching processes. Ann Math Stat. 1948;19(4):474-94.
7. Prasanth CB, Sandhya E. A generalized discrete uniform distribution. J Stat Appl Probab. 2012;5(1):1-13. https://doi.org/10.18576/jsap/050110
8. Sandhya E, Prasanth CB. Harris geometric distribution. Int J Innov Res Sci Eng Technol. 2012;5(2):2041-2052.
9. Sandhya E, Sherly S, Raju N. Harris family of discrete distributions. In: Proceedings of the National Seminar Conducted by Kerala Statistical Association and the Department of Statistics, University of Kerala, 2006.
10. Sandhya E, Sherly S, Jose KK, Raju N. Characterizations of the extended geometric, Harris, negative binomial, and gamma distributions. Stars Int. J, 2006.
11. Feller W. An introduction to probability theory and its applications. Vol. 2. Wiley, 1966.
12. Johnson NL, Kotz S, Balakrishnan N. Continuous univariate distributions. Vol. 1. Wiley; 1994.
13. Omari ALAI, Dobbah SA. On the mixture of Shanker and gamma distributions with applications to engineering data. J Radiat Res Appl Sci. 2023;16(1).
14. Rao AK, Pandey H. Parameter estimation of area-biased Ailamu-Jia distribution. Int J Phys Math Sci. 2020;10:218. http://www.cibtech.org/jpms.htm
15. Fazal A. Area-biased Poisson exponential distribution with applications. Biom Biostat Int J. 2018;7(3):256-61. https://doi.org/10.15406/bbij.2018.07.00216
16. Chouia S, Zeghdoudi H, Raman V, Beghriche A. A new size-biased distribution with application. J Appl Probab Stat. 2021;16(1):111-125. https://japs.isoss.net/April21.htm
17. Nwikpe BJ, Essi ID. Two-Parameter Nwikpe (TPAN) distribution with application. Asian J Probab Stat. 2021;12(1):56-67. https://doi.org/10.9734/ajpas/2021/v12i130279
18. Roshni C, Venkatesan D, Prasanth CB. A generalized area-biased power Ishita distribution: Properties and applications. Indian J Sci Technol. 2024;17(29):3037-43. https://doi.org/10.17485/IJST/v17i29.151
19. Anakha KK, Chacko VM. DUS-Kumaraswamy distribution: A bathtub-shaped failure rate model. Int J Stat Reliab Eng. 2021;8(3):359-367.
20. Gauthami P, Chacko VM. DUS transformation of inverse Weibull distribution: An upside-down failure rate model. Reliab Theory Appl. 2021;16(2):58-71.
21. Thomas, Chacko VM. Power generalized DUS transformation of exponential distribution. Int J Stat Reliab Eng. 2021;8(3):359-367.
22. Thomas, Chacko VM. Power generalized DUS transformation in Weibull and Lomax distributions. Reliab Theory Appl. 2023;18(1):368-384.