Quasi range dependent weighted classes of linear slope estimators

Introduction

Regression analysis serves as a statistical tool to explore the relationship between predictor (independent) variables and response (dependent) variable. It aims to envisage the value of the dependent variable based on independent variables. The SLR explores linear relationship between a single predictor variable and a response variable. It is established by the linear equation given by 𝑦 𝑖 = 𝛼 + 𝛽𝑥 𝑖 + 𝑒 𝑖 , 1 ≤ 𝑖 ≤ 𝑛 (1) where 𝛼 is intercept parameter, 𝛽 is slope parameter, 𝑦 𝑖 is response variable, 𝑥 𝑖 is predictor variable, 𝑒 𝑖 is independent and identically distributed (iid) random error from continuous distribution with distribution function 𝐹(•) having zero mean and finite variance 𝜎 2 . The goal is to estimate 𝛼, 𝛽 that best fit the observed data and make predictions based on this relationship. The slope parameter, 𝛽 quantifies the average variation in 𝑦 in response to a unit change in 𝑥.

Numerous estimators exist in the statistical literature to estimate 𝛽 with least square (𝑙𝑠) estimator due to Legendre (1805) [10] and Gauss (1809) [9] being first of its kind. It is given by

and its variance is given by

where 𝑥̅ = Under the assumption of equal distances (say 𝑑) of 𝑥 𝑖 𝑠, among the methods proposed by Bose (1938) [5] to estimate 𝛽, the method of differences at half range that outperforms all other estimators is given by

Theil (1950) [15] proposed an estimator for 𝛽 based on the median of pairwise slopes 𝑏 𝑖 = 𝑦 𝑗 -𝑦 𝑖 𝑥 𝑗 -𝑥 𝑖 , 1 ≤ 𝑖 < 𝑗 ≤ 𝑛 resulting in 𝑛(𝑛-1) 2

slopes. This method known as Theil's complete method is robust against outliers. Olkin and Yitzhaki (1992) [11] and Birkes and Dodge (1993) [4] discussed on 𝛽 ̂𝑜𝑦 , incorporating weights into the slopes to refine this estimation process. By using the weights, 𝑤 𝑖 = (𝑥 𝑗 -𝑥 𝑖 ) 2 ∑(𝑥 𝑗 -𝑥 𝑖 ) 2 , they showed that the weighted estimator 𝛽 ̂= ∑ 𝑤 𝑖 𝑏 𝑖 reduces to the least squares estimator. Cliff and Billy (2017) [7] proposed an estimator for 𝛽 using simple average method which is based on the average of successive slopes,

(2020) [12] simplified this procedure and investigated its performance. Singthongchai et. al. (2021) [14] improved the simple average method to estimate 𝛽. Bhat and Bijjargi (2023) [1] generalized Bose's estimators along with some modified estimators when 𝑥 𝑖 𝑠 have unequal distances. Among their estimators, 𝛽 ̂𝑢𝑑 based on quasi ranges has better performance than others and is equivalent to 𝛽 ̂𝑢ℎ based on half range. The estimator 𝛽 ̂𝑢𝑑 is given by

Here, 𝑞 𝑖 = 𝑥 (𝑛-𝑖) -𝑥 (𝑖+1) , 𝑖 = 1, 2, ⋯ 𝑚 -1 is 𝑖 𝑡ℎ quasi range among a set of observations (𝑥 𝑗 , 𝑦 𝑗 ), 𝑗 = 1, ⋯ , 𝑛 arranged in ascending order based on 𝑥 𝑗 , 𝑥 (𝑗) being 𝑗 𝑡ℎ order statistic and 𝑦 𝑗 * being the corresponding 𝑦 value and 𝑛 = 2𝑚.

Further, 𝑞 0 = 𝑥 (𝑛) -𝑥 (1) is the range of 𝑛 observations. Bhat and Bijjargi (2024a) [2] proposed adaptive 𝛽 ̂𝑢𝑑 estimators using various kinds of weights. Also, Bhat and Bijjargi (2024b) [3] developed two classes of estimators based on ratio of weighted average and weighted average of ratio of 𝑦 observations to the quasi ranges of 𝑥 observations. However, these weights do not account for magnitude of the 𝑥 observations which can restrict the effect of original value of 𝑥.

In this paper, we propose the classes of adaptive estimators of 𝛽 based on weighted average of quasi ranges and that of slopes obtained using quasi ranges with weights too depending on quasi ranges of predictor variables. The proposed classes of estimators are given in section 2. Their mean and variance are derived in section 3. The special cases of the proposed classes are discussed in section 4. Section 5 contains comparison among some members of these classes of estimators and are illustrated through examples in section 6. We record our conclusions based on observations in section 7.

Classes of adaptive linear slope estimators

The weighted average is a measure of central tendency. The estimators due to Bhat and Bijjargi (2024a and 2024b) [23] are assigned with the weights that are independent of actual 𝑥 values relying solely on the position of quasi range. The assignment of weights remains uniform across all datasets, irrespective of the magnitude of 𝑖 𝑡ℎ quasi range which is again dependent on 𝑥 values. This means that, regardless of whether the dataset comprises of high-magnitude or lowmagnitude values, the weights assigned remain constant and lacks sensitivity to the magnitude of the observations. On similar lines to the development of 𝛽 ̂𝑜𝑦 which has varying weights to account for data variability, the proposed method also assigns varying weights. However, unlike this method, that requires calculating

slopes and their weighted average, the proposed method simplifies the computation by requiring only 𝑛 2 slopes and their weighted average. This reduction in computational complexity makes the estimator more feasible with maintaining robustness and improvement in the overall performance. Additionally, 𝛽 ̂𝑜𝑦 considers all pairwise slopes which do not effectively distinguish between extreme and middle observations. But, incorporating the magnitude of distances obtained from quasi ranges as weights, the proposed classes of estimators aim to reflect the true variability and spread within the data distinguishing extreme and central observations. Hence, the proposed method enables a customized weighting system that prioritizes observations based on their positions in the dataset. This leads to a representative and accurate slope estimates. In this section, we introduce two distinct classes of adaptive linear slope estimators viz. 𝛽 ̂1𝐷 𝑘 and 𝛽 ̂2𝐷 𝑘 for estimating 𝛽 in SLR model. Both the classes are based on quasi ranges of 𝑥 variable and their weights also consider the quasi ranges of 𝑥 variable.

The weighting scheme in proposed classes of estimators is motivated by a desire to enhance efficiency, robustness and adaptability in estimation of 𝛽. The emphasis on extreme or middle observations is modulated by the value 𝑘, allowing for a flexible weighting scheme that adapts to the characteristics of the data. The weight assigned in both the classes is determined by the expression 𝑤 𝑖 𝑘 = 𝑞 𝑚-𝑖 𝑘 . Since the 𝑥 𝑖 values are ordered, all the 𝑞 𝑚-𝑖 are positive and increases as 𝑖 goes from 1 to 𝑚. The exponent 𝑘 is instrumental in defining the weighting scheme. For 𝑘 > 0, the weights assigned to the quasi ranges increase with 𝑖. That is, 𝑞 𝑚-𝑖 obtained from extreme 𝑥 𝑖 observations (those farther from the median) will have higher weights. For 𝑘 < 0, the weights assigned to the quasi ranges decrease with 𝑖. That is, 𝑞 𝑚-𝑖 derived from observations closer to the median will have higher weights which enhances robustness against outliers. When predictor variables are equidistant,

Mean and variance of proposed classes of estimators

In this section, the mean and variance of proposed classes of estimators are obtained. The mean of class 𝛽 ̂1𝐷 𝑘 is given by

and the mean of 𝛽 ̂2𝐷 𝑘 is given by

= 𝛽

Both 𝛽 ̂1𝐷 𝑘 and 𝛽 ̂2𝐷 𝑘 admit unbiased estimators of 𝛽. The variance of 𝛽 ̂1𝐷 𝑘 is given by

When 𝑥 (𝑖) 𝑠 are equidistant, 𝑉(𝛽 ̂1𝐷 𝑘 ) can be written as,

) ∵ 𝑤 𝑖 = 𝑞 𝑚-𝑖 (11) When 𝑥 (𝑖) ′𝑠 are equidistant,

Also, when 𝑥 (𝑖) ′𝑠 are evenly spaced, the least square estimator given in (2) reduces to

12 (13) and its variance is given by

As variance expressions of two classes of proposed estimators are in complex forms, we obtain optimum 𝑘 that yields minimum variances in ( 10) and ( 12) using a derivative-free optimization algorithm due to Brent (1971) [6] . This numerical method is widely known for its robustness and efficiency, often outperforming other optimization algorithms in terms of convergence speed and accuracy. The minimum variances of 𝛽 ̂1𝐷 𝑘 and 𝛽 ̂2𝐷 𝑘 through Brent's algorithm are obtained as per steps given below.

Step Step 3: Function evaluation and interval update 𝑓(𝑘) is evaluated and the points are updated as per 𝑓(𝑘) < 𝑓(𝑏) or 𝑓(𝑘) ≥ 𝑓(𝑏).

Step 4: Convergence check

The process is repeated until the width of an interval |𝑏 -𝑎| < 𝜖, that is, the function value at the trial point becomes smaller than a predefined tolerance |𝑓(𝑘)| < 𝜖 and the maximum number of iterations 𝑁 is reached.

Step 5: Output Once the convergence is achieved, the value of 𝑘 corresponding to the minimum of the function 𝑓(𝑘) is returned within the refined interval [𝑎, 𝑏]. Thus, the value of 𝑘 that minimizes 𝑉(𝛽 ̂1𝐷 𝑘 ) and 𝑉(𝛽 ̂2𝐷 𝑘 ) is achieved and we observe that, the minimum value of 𝑉(𝛽 ̂1𝐷 𝑘 ) is at 𝑘 = 1 and 𝑉(𝛽 ̂2𝐷 𝑘 ) is at 𝑘 = 2.

In Figure 1, we plot 𝑉(𝛽 ̂1𝐷 𝑘 ) and 𝑉(𝛽 ̂2𝐷 𝑘 ) given in ( 10) and ( 12) for 𝑑 = 1, 𝜎 = 1 and various values of 𝑚 and 𝑘. The Figure 1 indicates that, the variance decreases as 𝑚 increases. Also, the point marked on the line denotes the minimum value of the variance observed for 𝛽 ̂1𝐷 1 and 𝛽 ̂2𝐷 2 .

Further, for 𝑘 = -2, -1, 0, 1, 2, 3 and various values of 𝑛, we furnish the computations of 𝑉(𝛽 ̂1𝐷 𝑘 ), 𝑉(𝛽 ̂𝑙𝑠 ), 𝑉(𝛽 ̂𝑒ℎ ) and 𝑉(𝛽 ̂2𝐷 𝑘 ) in Table 1 and Table 2 respectively.

Some special cases of the classes of estimators

Results 1: and its variance is given by

When 𝑥 (𝑖) ′𝑠 are equidistant, 𝛽 ̂1𝐷 1 reduces to

and its variance is given by

Proceeding on similar lines, we get 𝑉(𝛽 ̂2𝐷 2 ) = 𝑉(𝛽 ̂𝑙𝑠 ).

Result 4:

For 𝑘 = -1 and 𝑘 ′ = 0, 𝛽 ̂1𝐷 𝑘 = 𝛽 ̂2𝐷 𝑘 ′ = 𝑏 ̅ .

On similar lines, we notice that, 𝛽 ̂2𝐷 0 = 𝑏 ̅ .

Performance of the proposed classes of estimators

In this section, we study the performance of the classes of estimators using relative efficiency (RE) and simulated data.

The RE serves as a quantitative indicator to assess the comparative precision of estimators. In the context of two unbiased estimators, 𝑇 1 and 𝑇 2 , 𝑅𝐸(𝑇 1 , 𝑇 2 ) is given by

The 𝑅𝐸(𝑇 1 , 𝑇 2 ) > 1 indicates that estimator 𝑇 1 offers better precision relative to estimator 𝑇 2 . From table 1, it is evident that, 𝑅𝐸(𝛽 ̂1𝐷 𝑘-1 , 𝛽 ̂2𝐷 𝑘 ) = 1. Hence, 𝑅𝐸(𝛽 ̂1𝐷 1 , 𝛽 ̂2𝐷 2 ) = 1. For various values of 𝑛 ≤ 200, 𝛽 ̂1𝐷 1 performs better than 𝛽 ̂1𝐷 0 by 25% to 34%. It is also better than 𝛽 ̂1𝐷 2 by 6% to 8% for 𝑛 < 100. Moreover, 𝛽 ̂1𝐷 1 shows 17% to 18% higher performance over 𝛽 ̂1𝐷 3 for 𝑛 < 100. For 𝑛 ≤ 200, 𝛽 ̂2𝐷 2 shows 17% to 19% higher efficiency than 𝛽 ̂2𝐷 0 . It exhibits about 2% to 3% higher efficiency than 𝛽 ̂2𝐷 1 . Moreover, 𝛽 ̂2𝐷 2 is approximately 7% to 9% more efficient than 𝛽 ̂2𝐷 3 . For 𝑛 > 200, 𝛽 ̂2𝐷 2 and 𝛽 ̂2𝐷 0 , 𝛽 ̂2𝐷 1 , 𝛽 ̂2𝐷 3 are equally efficient.

To evaluate the robustness and effectiveness of the proposed classes of estimators, we generate the simulated data. The random samples of various size 𝑛 is generated for independent variable𝑥, incorporating various percentages of outliers to emulate real scenarios. The true values of 𝛼 and 𝛽 are fixed at 2 and 3, respectively, while errors are introduced from a normal distribution with mean 0 and variance 1.

Employing the simple linear regression model with these predetermined values, corresponding 𝑦 observations are computed for different sample sizes and percentages of outliers in 𝑥. Subsequently, the proposed estimator is applied to the generated data to estimate the slope parameter 𝛽. In this study, we analyze only the performance of 𝛽 ̂2𝐷 𝑘 which is equal to 𝛽 ̂1𝐷 𝑘-1 . By fixing the true value of 𝛽 at 3, we analyze how closely the estimated 𝛽 values match this true value. The results are tabulated for various values of 𝑛, 𝑘 and percentage of outliers introduced for 𝑥 observations in Table 3. Here, 𝛽 ̂𝑙𝑠 = 0.321600 and 𝑉(𝛽 ̂𝑙𝑠 ) = 0.0000204𝜎 2 . The computed values of 𝛽 ̂1𝐷 𝑘 , 𝛽 ̂2𝐷 𝑘 and their variances are given in Table 5 and regression lines are plotted in Figure 2. When an outlier is induced, the computed values of 𝛽 ̂1𝐷 𝑘 , 𝛽 ̂2𝐷 𝑘 and their variance is furnished in Table 6 and regression lines are plotted in Figure 3. Also, 𝛽 ̂𝑙𝑠 = 0.080710 with variance 𝑉(𝛽 ̂𝑙𝑠 ) = 0.000009 𝜎 2 . From Table 5 and Figure 2, it is observed that, 𝛽 ̂1𝐷 𝑘 , 0 ≤ 𝑘 ≤ 2, 𝛽 ̂2𝐷 𝑘 , 1 ≤ 𝑘 ≤ 2, 𝛽 ̂𝑙𝑠 have near values to the slope with overlapping regression lines and 𝛽 ̂2𝐷 0 results in poor fit. As 𝑘 > 0 effectively captures data patterns, further exploration has limited benefits. After introducing an outlier in the dataset and plotting regression lines based on 𝛽 ̂1𝐷 𝑘 , 𝛽 ̂2𝐷 𝑘 , -2 ≤ 𝑘 ≤ 2, from Table 6 and Figure 3, it is seen that, there is a notable good fit of regression lines to the data. Specifically, for 𝑘 ≤ 0, the estimator exhibits increased robustness against outlier leading to improved accuracy in estimating the slope parameter 𝛽.

Conclusions

• Two classes of estimators 𝛽 ̂1𝐷 𝑘 and 𝛽 ̂2𝐷 𝑘 which utilize the weights based on quasi ranges of predictor variable are proposed for slope parameter, 𝛽 in SLR model. • One of the classes is based on ratio of sum of the weighted deviations of corresponding 𝑦 observations to sum of the weighted quasi ranges of predictor variable (𝑥) and other is weighted average of slopes obtained using quasi ranges. • The proposed classes utilize the weights depending on quasi ranges of 𝑥 emphasizing the magnitude of the observations. • The classes of estimators admit unbiased and consistent estimators for 𝛽. • The proposed weighting scheme effectively distinguishes between extreme and central observations. Hence, allows for a customized weighting system that prioritizes observations based on their positions within the dataset. • The exponent 𝑘 in weighting scheme introduces flexibility in intensifying the customization. • The optimal value of 𝑘 (say 𝑘 * ) for the proposed classes of estimators obtained using Brent's method by minimizing their variances are 𝑘 * = 1 and 𝑘 * = 2 for 𝛽 ̂1𝐷 𝑘 and 𝛽 ̂2𝐷 𝑘 respectively. Also, 𝛽 ̂1𝐷 𝑘-1 = 𝛽 ̂2𝐷 𝑘 .

• 𝛽 ̂1𝐷 0 , 𝛽 ̂2𝐷 1 reduces to the estimator due to Bhat and Bijjargi (2023) [1] . Under equidistant 𝑥 𝑖 ′𝑠 they reduce to the estimator due to Bose (1938) [5] . • The variances of 𝛽 ̂1𝐷 1 and 𝛽 ̂2𝐷 2 are equal to the variance of least square estimator when 𝑥 𝑖 ′𝑠 have equal distance. • For 𝑘 > 0, both the classes of estimators have heavier weights to extreme observations and for 𝑘 < 0, they have heavier weights to middle observations. • The members of classes of estimators show equal performance for large samples, whereas, 𝛽 ̂1𝐷 1 and 𝛽 ̂2𝐷 2 are better than other members of their respective classes for smaller samples.

References

1. Bhat SV, Bijjargi SM. Nonparametric approach to estimation in linear regression. International Journal of Current Research. 2023;15(04):24419-24424.
2. Bose SS. Relative efficiencies of regression coefficients estimated by method of finite differences. Sankhya. 1938;3:339-346.
3. Brent RP. An algorithm with guaranteed convergence for finding a zero of a function. The computer journal. 1971;14(4):422-425.
4. Cliff KR, Billy KM. Estimation of the parameters of a linear regression system using the simple averaging method. Global Journal of Pure and Applied Mathematics. 2017;13(11):7749-7758.
5. Prabowo A, Sugandha A, Tripena A, Mamat M, Sukono, Budiono R. A New Method to Estimate Parameters in the Simple Regression Linear Equation. Mathematics and Statistics. 2020;8(2):75-81.
6. Singthongchai J, Thongmual N, Nitisuk N. An Improved Simple Averaging Approach for Estimating Parameters in Simple Linear Regression Model. Mathematics and Statistics. 2021;9(6):939-946.
7. Theil H. A rank-invariant method of linear and polynomial regression analysis. Indagationes Mathematicae. 1950;12(85):85-173.