Performance properties of disturbance variance estimation using the precautionary loss function

Introduction

In statistical inference, the use of loss functions is quite valuable in minimizing the risk of the estimator and can affect estimators' properties and rankings. In general, the symmetrical loss functions are frequently employed in evaluating risk i.e., Squared error loss function, used by various researchers [Ohtani (1987, 2001) [12,15] , Matta and Casella (1990) [10] , James, & Stein (1992)] [7] . In some situations, where positive and negative errors have differing effects then employment of asymmetric loss function has been most justified for e.g. LINEX loss function. Norstrom (1996) [11] introduced an alternative asymmetric precautionary loss function, and also presented a general class of precautionary loss functions as a special case. These loss functions approach infinitely near the origin to prevent underestimation, thus giving conservative estimators, especially when underestimation may lead to serious consequence (Ali et al. 2012) [1] . Precautionary loss has captured enormous attention and several authors have used this loss function in different contexts, specifically in Bayesian context Huda & Aref, (2016) and Selvam and Geetha (2023) [6,15] . In the linear regression model sample information used to estimate unknown parameters. In some situations, prior information in the form of linear restrictions is available to estimate unknown parameters. By combining this information with sample information several improved estimators for regression coefficient and for variance were proposed. [see e.g. Srivastava and Rao (1992) and Dube, M., & Singh, V. (1995)]. Dube and Chandra (2007) [3,4,14] examined the effect of Stein rule estimator of the disturbance variance when disturbances are normally distributed under the mean squared error criterion. Chandra and Tyagi (2015) [2] obtained the approximate expression for the risks of the Stein rule and the usual estimator for regression error variance under this function. Dube et al. (2015) [5] examined the performance properties of the disturbance variance estimator under the LINEX loss function. Recently Kumar (2020), (2022) [8,9] analyzed the Distribution and studied the performance properties of pre-test estimator of disturbance variance in linear regression model. Here, we will investigate properties of the usual estimator of disturbance variance estimator and restricted error variance under precautionary loss function. The plan of the paper is as follows: Section 2 describes the model and the estimators. Section 3 provides the risk function of the estimators of disturbance variance under the precautionary loss function. The risk comparison of the estimators of disturbance variance, results and conclusion is discussed for different parameters in Section 4. In appendix, a brief derivation of the risk function of estimators under the precautionary loss function is given.

The model and the estimators

Consider the standard linear regression model.

where Y is a vector that includes information on the response variable, 𝑋 is a matrix of observations on the explanatory variables, 𝛽 is a vector of unknown parameters related to the regressors, and 𝜀 is a vector of disturbances. It is assumed that the components of 𝜀 are independently and identically distributed, each having a normal distribution with a variance of 𝜎 2 and a mean of zero. 𝐸(𝜀) = 0, 𝐸(𝜀𝜀 ′ ) = 𝜎 2 𝐼 𝑛 , where 𝜎 2 Is finite and unknown. The classical estimation of the linear regression model makes use of the least squares estimation procedure for the estimation of the coefficients in the model. When estimating, least squares approaches produce the best linear unbiased estimators, meaning that the resulting estimator is easily understandable and has the least variance around the true parametric value.

Using the principle of least squares, the following estimator is used.

Consider a scenario in which the linear regression model has constraints on the parameters based on prior knowledge about one or more of the parameters. This prior information could be used to improve estimators' quality. If this earlier data is accurate, then the model's linear constraints on the coefficients as

where 𝑅 is a 𝑞 × 𝑝, (𝑞 < 𝑝) matrix of known elements with rank 𝑞, 𝑞 being the number of constraints applied to the coefficients and 𝑟 is 𝑞 × 1 vector of known elements. The restricted least squares estimator of β using (2.1) and (2.3) is given by

The usual and restricted estimator of error variance is one entirely based on the sample information, and the other utilizes both sample and prior information. The following estimators are constructed.

Here 𝑒 and 𝑒 𝑅 the residual vector obtained using ordinary least squares and the restricted least squares estimator, respectively, and 𝑚 and 𝑚 ′ are the arbitrary scalars characterizing the estimators given by 𝑒 = 𝑌 -𝑋𝑏, 𝑒 𝑅 = 𝑌 -𝑋𝑏 𝑟 , 𝑚 = 𝑣 + 𝜃, 𝑚 ′ = 𝑣 + 𝑞 + 𝜃, where 𝑣 = 𝑛 -𝑝 is a degree of freedom.

Similarly, for 𝑠 𝑅 2 given by

(𝑣+𝑞+𝜃) 2 (𝜃 2 + 2(𝑣 + 𝑞))

Risk of variance estimator under precautionary loss function

The precautionary loss function is expressed by 𝐿(𝛼 ̂, 𝛼 ) = (𝛼-𝛼 ̂)2 𝛼 ̂, where 𝛼 is population parameter is estimated by 𝛼 ̂. To estimate 𝜎 2 by 𝑠 2 and 𝑠 𝑅 2 under this function, the following expression will be solved

It is easy to compute optimum value of 𝜃 denoted by 𝜃 𝑜𝑝𝑡 from (3.4) which is given by

It can be easily verified that for 𝑞 = 0, 𝜆 = 0 the risk of the restricted estimator reduces to the usual estimator of error variance which is given in (3.3) and optimum value for usual estimator is

Numerically computation, results and conclusion

The performance of estimators assessed under different conditions numerically for different parameters 𝜆, 𝜃, 𝑞 and 𝑣. The numerical results reveal the restricted estimator 𝑠 𝑅 2 has lower risk compared to usual estimator 𝑠 2 under the precautionary loss function from Table 1-3. The usual estimator 𝑠 2 and the restricted estimator 𝑠 𝑅 2 of error variance are unbiased estimators for 𝜃 = 0 and have minimum MSE for 𝜃 = 2, therefore, the performance of these estimators is also analyzed for 𝜃 = 0, 𝜃 = 2 and for optimum value (𝜃 𝑜𝑝𝑡 ) under precautionary loss function. The behavior of the risk of 𝑠 𝑅 2 when 𝜎 2 = 1, is summarized in figures 1-10, while the number of restrictions (𝑞) and the degrees of freedom (𝑣) vary. Figures 1 and2 provide the risk of 𝑠 𝑅 2 for 𝜃 = 0, while Figures 3 and4 correspond to 𝜃 = 2, and Figures 5 and6 illustrate the risks associated with the optimum value (𝜃 𝑜𝑝𝑡 ). From these figures, it is evident that the risks at 𝜃 = 0 and 𝜃 = 2 increase as the non-centrally parameter 𝜆 increases. The risk at the optimal value of 𝜃, on the other hand, decreases as 𝜆 increases. The optimum choice offers higher stability, but fixed choices of 𝜃 are more susceptible to variations in 𝜆. An additional trend in these figures is that risk decreases as the number of restrictions increases, which indicates the efficiency gained by the incorporation of additional information. Moreover, as the degrees of freedom become larger, the risk functions for different values of 𝑞 nearly coincide, suggesting that the effect of restrictions becomes less pronounced as the sample size or information increases. Figures 7 and8 further demonstrate the risks for 𝜃 = 0, 𝜃 = 2, and 𝜃 𝑜𝑝𝑡 when 𝑞 = 1 and 𝑞 = 5 with 𝑣 = 10. Similarly, Figures 9 and10 shows the same patterns when 𝑣 = 50. It is evident from these results that the risk is always minimal at the optimal value, 𝜃 𝑜𝑝𝑡 , and that it stabilizes as 𝜆 increases. When comparing 𝜃 = 0 and 𝜃 = 2, the point for alteration is approximately at 𝜆 = 1. There is a smaller risk at 𝜃 = 0 below this value and at 𝜃 = 2 higher. Overall, these findings suggest that the optimal value, 𝜃 𝑜𝑝𝑡 , consistently offers the lowest and most stable risk across all situations, and the choice of 𝜃 should be based on the magnitude of the non-centrality parameter 𝜆. This shows that the optimal choice of 𝜃 is more reliable than fixed selections of 𝜃.

References

1. Ali S, Aslam M, Abbas N, Kazmi SMA. Scale parameter estimation of the Laplace model using different asymmetric loss functions. Int J Stat Probab. 2012;1(1):105.
2. Chandra S, Tyagi D. Estimation of error variance under the LINEX loss function. Int J Math Sci Appl. 2015;5(2).
3. Dube M, Singh V. Predictive efficiency of improved estimators in restricted regression models. Test. 1995;4:323-331.
4. Dube M, Chandra S. A note on inadmissibility of the iterative Stein-rule estimator of the disturbance variance. J Indian Soc Agric Stat. 2007;61(3):341-343.
5. Dube M, Kumar V. The performance properties of disturbance variance under restrictions using LINEX loss function. J Appl Probab. 2015;10(2):79-88.
6. Huda AR, Aref R. Reliability estimation in inverse Rayleigh distribution using precautionary loss function. Math Stat J. 2016;2(3):9-15.
7. James W, Stein C. Estimation with quadratic loss. In: Breakthroughs in statistics: Foundations and basic theory. New York: Springer; 1992. p. 443-460.
8. Kumar V. Analyzing the distribution of pre-test estimator of error variance in linear regression model. Afr Diaspora J Math. 2020;23(8):7485.
9. Kumar V. Assessing the performance properties of pre-test estimator of disturbance variance in linear regression model. Afr Diaspora J Math. 2022;25(2):68-83.
10. Matta JM, Casella G. Developments in decision-theoretic variance estimation. Stat Sci. 1990;5(1):90-120.
11. Norstrom JG. The use of precautionary loss functions in risk analysis. IEEE Trans Reliab. 1996;45(3):400-403.
12. Ohtani K. MSE dominance of the pre-test iterative variance estimator over the variance estimator in regression. Stat Probab Lett. 2001;54:331-340.
13. Selvam R, Geetha M. Characterization of Bayes risk under different loss functions using informative priors. Int J Eng Res Dev. 2023;19(2):16-24.
14. Srivastava VK, Bhaskara Rao B. Estimation of disturbance variance in linear regression models under asymmetric loss function. J Quant Econ. 1992;8:341-345.
15. Ohtani K. Inadmissibility of the iterative Stein-rule estimator of the disturbance variance in linear regression. Econ Lett. 1987;24(1):51-55.