Preference for a class of super-efficient estimators of the normal mean: A study on sample size requirement

Introduction

The notion of 'super-efficient estimator' flowed out of an unpublished work of Hodges (1951) [4] which was followed up with theoretical developments by Basu (1952) [2] , Le Cam (1953, 1956, 1960, 1972) [5,6,7,8] and Stein (1956) [11] . There has not been much development on this topic ever since, except for a few sporadic research articles by Bahadur (1983) [1] , Sethuraman (2004) [9] and Durairajan (2012) [3] . Even though the above-mentioned authors addressed the problem of obtaining super-efficient estimators in different contexts, there were no systematic approaches taken towards constructing super-efficient estimators. Recently, Sivasakthi et. al. (2017) [10] addressed this problem in the context of estimating a real parameter through the 'Delta Method' of asymptotic inference theory and discussed a number of illustrative examples. The property of super-efficiency is asymptotic and a super-efficient estimator (SEE) compares favourably with the asymptotically efficient estimator for adequately large samples. The question that naturally arises in this context is about the sample size that would be required to prefer a super-efficient estimator over the efficient estimator. It is found that there has been no attempt till now to address this question. Algebraic closed-form expressions are not available for the 'Mean-Square Error' or 'Variance' of the super-efficient estimators and, so comparisons are to be done numerically. This paper is an exploration in the above direction. In the recent work of Sivasakthi et al. (2017) [10] , the super-efficient estimation of the mean of a normal population with unit variance, N(θ, 1), was considered as one of the applications of the 'Delta Method' of deriving super-efficient estimators. For this situation, a class of superefficient estimators was constructed by Sivasakthi et. al. (2017) [10] with super-efficiency at θ= 0. But, the question on the sample size required for the SEE to overtake the MLE remains unanswered. In this paper, the important question on sample size needed to prefer the SEE over the MLE is addressed by a numerical study. As SEE's are only asymptotically unbiased, the comparison is based on mean-square errors instead of variances of the estimators. For larger sample sizes, however, the mean-square error is approximately same as variance owing to asymptotic unbiasedness. This paper contains a total of five sections including the present introduction section. Section 2 reviews the class of super-efficient estimators under consideration for estimating the mean of a normal distribution with unit variance.

Section 3 provides the theoretical results concerning the mean-square error of the SEE. Section 4, presents the results of the numerical work which have been carried out using 'R' language. Finally, Section 5 contains concluding remarks with recommendations on the sample sizes needed for the preferential use of the class of SEE's over MLE.

A class of super-efficient estimators of the normal mean

As stated earlier, in a recent work, Sivasakthi et. al. (2017) [10] applied the 'Delta Method' to obtain super-efficient estimators for the mean of a normal distribution with unit variance, vizN(θ, 1), .

R  

One class of super-efficient estimators considered in that paper is given by

where k> 1 is any known positive integer. This class is generated by the choice of the function g(θ) = k  on the parameter space. The estimators in this class are superefficient at θ= 0 and we refer to Sivasakthi et. al. (2017) [10] for other details. Other classes of super-efficient estimators for the same family of normal distributions have also been proposed in that paper. One class is specified by

is a chosen point and 1 a < 1.This is a generic choice applicable to any distribution, not just N(θ, 1) alone. Yet another class is generated by

But the class of estimators in (2.1) is interesting in that, when the sample mean is closer to zero, which happens with higher likelihood when the true parameter is zero or close to zero, the super-efficient estimator 'improves' the estimate by moving closer to zero by raising it to an integer power 'k'. Thus, there is a rationale in considering the class of estimators in (2.1). The use of a super-efficient estimator from the class instead of the efficient estimator is therefore recommended for adequately large samples. The interesting question of how large the sample should be to prefer a super-efficient estimator is sought to be addressed in the subsequent sections of this paper.

Some Theoretical Results

We provide some theoretical results on the class of S.E.E.'s ) (k n   in this section towards deriving an expression for the Mean Square Error of the estimators.

Bias of SEE

3.2: Comparison of SEE's with MLE

Gain in preferring the SEE's over the MLE

The MLE of θ is X which is efficient with variance 1/n. The gain or loss in preferring ) (k

Numerical Investigation on the performance of S.E.E. over M.L.E.

As seen in the previous section,

does not have a closed form expression and the process of finding the required sample sizes 'n' cannot be resolved mathematically, but only numerically. Further, even though the class (2.1) itself is infinitely large, we take a small subset of the class by choosing k from 2 to 5 and do the numerical investigation in the sequel.

As pointed out earlier, the comparison of the SEE's

with the Asymptotic Efficient estimator (MLE) X is through the 'Gain' function found in (3.9). We seek to know the 'n' values for which the 'Gain Function' is non-negative. Along with this, the Bias in

is also of interest because the estimator is not exactly unbiased and is only asymptotically so. Thus, in this section, we shall present the MSE, Bias and the Gain in

We have carried out the computations of these measures for a very wide choice of sample sizes, namely, n = 1 (1) 10 ( 10) 100 (100) 1000 (1000) 10000 (10000) 100000 and for a good range of θ values, namely, θ = -10 (1) -1 (0.1) 1 (1) 10. However, in order to keep the length of this article reasonably moderate, we present the results for k = 2 only with a few selected choices of θ values, namely θ = -10, -5, -1, -0.5, -0.1, 0, 0.5, 1, 3, 6. The sample sizes 'n' required for preferring the SEE We need to mention here that the results have been rounded off to 5 decimal places as the largest sample size that we have considered is 100000 and the variance of the MLE X corresponding to this sample size is 10 -5 . 0.00500 0.00000 0.00000 200 0.00500 0.00000 0.00000 300 0.00333 0.00000 0.00000 300 0.00333 0.00000 0.00000 400 0.00250 0.00000 0.00000 400 0.00250 0.00000 0.00000 500 0.00200 0.00000 0.00000 500 0.00200 0.00000 0.00000 600 0.00167 0.00000 0.00000 600 0.00167 0.00000 0.00000 700 0.00143 0.00000 0.00000 700 0.00143 0.00000 0.00000 800 0.00125 0.00000 0.00000 800 0.00125 0.00000 0.00000 900 0.00111 0.00000 0.00000 900 0.00111 0.00000 0.00000 1000 0.00100 0.00000 0.00000 1000 0.00100 0.00000 0.00000 2000 0.00050 0.00000 0.00000 2000 0.00050 0.00000 0.00000 3000 0.00033 0.00000 0.00000 3000 0.00033 0.00000 0.00000 4000 0.00025 0.00000 0.00000 4000 0.00025 0.00000 0.00000 5000 0.00020 0.00000 0.00000 5000 0.00020 0.00000 0.00000 6000 0.00017 0.00000 0.00000 6000 0.00017 0.00000 0.00000 7000 0.00014 0.00000 0.00000 7000 0.00014 0.00000 0.00000 8000 0.00012 0.00000 0.00000 8000 0.00013 0.00000 0.00000 9000 0.00011 0.00000 0.00000 9000 0.00011 0.00000 0.00000 10000 0.00010 0.00000 0.00000 10000 0.00010 0.00000 0.00000 20000 0.00005 0.00000 0.00000 20000 0.00005 0.00000 0.00000 30000 0.00003 0.00000 0.00000 30000 0.00003 0.00000 0.00000 40000 0.00002 0.00000 0.00000 40000 0.00003 0.00000 0.00000 50000 0.00002 0.00000 0.00000 50000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 70000 0.00001 0.00000 0.00000 70000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 0.00333 0.00000 0.00000 300 0.00333 0.00000 0.00000 400 0.00250 0.00000 0.00000 400 0.00250 0.00000 0.00000 500 0.00200 0.00000 0.00000 500 0.00200 0.00000 0.00000 600 0.00167 0.00000 0.00000 600 0.00167 0.00000 0.00000 700 0.00143 0.00000 0.00000 700 0.00143 0.00000 0.00000 800 0.00125 0.00000 0.00000 800 0.00125 0.00000 0.00000 900 0.00111 0.00000 0.00000 900 0.00111 0.00000 0.00000 1000 0.00100 0.00000 0.00000 1000 0.00100 0.00000 0.00000 2000 0.00050 0.00000 0.00000 2000 0.00050 0.00000 0.00000 3000 0.00033 0.00000 0.00000 3000 0.00033 0.00000 0.00000 4000 0.00025 0.00000 0.00000 4000 0.00025 0.00000 0.00000 5000 0.00020 0.00000 0.00000 5000 0.00020 0.00000 0.00000 6000 0.00017 0.00000 0.00000 6000 0.00017 0.00000 0.00000 7000 0.00014 0.00000 0.00000 7000 0.00014 0.00000 0.00000 8000 0.00012 0.00000 0.00000 8000 0.00012 0.00000 0.00000 9000 0.00011 0.00000 0.00000 9000 0.00011 0.00000 0.00000 10000 0.00010 0.00000 0.00000 10000 0.00010 0.00000 0.00000 20000 0.00005 0.00000 0.00000 20000 0.00005 0.00000 0.00000 30000 0.00003 0.00000 0.00000 30000 0.00003 0.00000 0.00000 40000 0.00002 0.00000 0.00000 40000 0.00002 0.00000 0.00000 50000 0.00002 0.00000 0.00000 50000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 70000 0.00001 0.00000 0.00000 70000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 θ = ̶ 0.1 0.01100 0.09740 -0.01080 5000 0.00000 0.00020 0.00020 6000 0.01030 0.09052 -0.01013 6000 0.00000 0.00017 0.00017 7000 0.00939 0.08191 -0.00925 7000 0.00000 0.00014 0.00014 8000 0.00833 0.07193 -0.00820 8000 0.00000 0.00012 0.00013 9000 0.00715 0.06115 -0.00704 9000 0.00000 0.00011 0.00011 10000 0.00594 0.05025 -0.00584 10000 0.00000 0.00010 0.00010 20000 0.00018 0.00108 -0.00014 20000 0.00000 0.00005 0.00005 30000 0.00003 0.00000 0.00000 30000 0.00000 0.00003 0.00003 40000 0.00003 0.00000 0.00000 40000 0.00000 0.00002 0.00003 50000 0.00002 0.00000 0.00000 50000 0.00000 0.00002 0.00002 60000 0.00002 0.00000 0.00000 60000 0.00000 0.00002 0.00002 70000 0.00001 0.00000 0.00000 70000 0.00000 0.00001 0.00001 80000 0.00001 0.00000 0.00000 80000 0.00000 0.00001 0.00001 90000 0.00001 0.00000 0.00000 90000 0.00000 0.00001 0.00001 100000 0.00001 0.00000 0.00000 100000 0.00000 0.00001 0.00001 θ = 0.5 0.00333 0.00000 0.00000 300 0.00333 0.00000 0.00000 400 0.00250 0.00000 0.00000 400 0.00250 0.00000 0.00000 500 0.00200 0.00000 0.00000 500 0.00200 0.00000 0.00000 600 0.00167 0.00000 0.00000 600 0.00167 0.00000 0.00000 700 0.00143 0.00000 0.00000 700 0.00143 0.00000 0.00000 800 0.00125 0.00000 0.00000 800 0.00125 0.00000 0.00000 900 0.00111 0.00000 0.00000 900 0.00111 0.00000 0.00000 1000 0.00100 0.00000 0.00000 1000 0.00100 0.00000 0.00000 2000 0.00050 0.00000 0.00000 2000 0.00050 0.00000 0.00000 3000 0.00033 0.00000 0.00000 3000 0.00033 0.00000 0.00000 4000 0.00025 0.00000 0.00000 4000 0.00025 0.00000 0.00000 5000 0.00020 0.00000 0.00000 5000 0.00020 0.00000 0.00000 6000 0.00017 0.00000 0.00000 6000 0.00017 0.00000 0.00000 7000 0.00014 0.00000 0.00000 7000 0.00014 0.00000 0.00000 8000 0.00012 0.00000 0.00000 8000 0.00012 0.00000 0.00000 9000 0.00011 0.00000 0.00000 9000 0.00011 0.00000 0.00000 10000 0.00010 0.00000 0.00000 10000 0.00010 0.00000 0.00000 20000 0.00005 0.00000 0.00000 20000 0.00005 0.00000 0.00000 30000 0.00003 0.00000 0.00000 30000 0.00003 0.00000 0.00000 40000 0.00002 0.00000 0.00000 40000 0.00002 0.00000 0.00000 50000 0.00002 0.00000 0.00000 50000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 70000 0.00001 0.00000 0.00000 70000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 0.00250 0.00000 0.00000 400 0.00250 0.00000 0.00000 500 0.00200 0.00000 0.00000 500 0.00200 0.00000 0.00000 600 0.00167 0.00000 0.00000 600 0.00167 0.00000 0.00000 700 0.00143 0.00000 0.00000 700 0.00143 0.00000 0.00000 800 0.00125 0.00000 0.00000 800 0.00125 0.00000 0.00000 900 0.00111 0.00000 0.00000 900 0.00111 0.00000 0.00000 1000 0.00100 0.00000 0.00000 1000 0.00100 0.00000 0.00000 2000 0.00050 0.00000 0.00000 2000 0.00050 0.00000 0.00000 3000 0.00033 0.00000 0.00000 3000 0.00033 0.00000 0.00000 4000 0.00025 0.00000 0.00000 4000 0.00025 0.00000 0.00000 5000 0.00020 0.00000 0.00000 5000 0.00020 0.00000 0.00000 6000 0.00017 0.00000 0.00000 6000 0.00017 0.00000 0.00000 7000 0.00014 0.00000 0.00000 7000 0.00014 0.00000 0.00000 8000 0.00013 0.00000 0.00000 8000 0.00012 0.00000 0.00000 9000 0.00011 0.00000 0.00000 9000 0.00011 0.00000 0.00000 10000 0.00010 0.00000 0.00000 10000 0.00010 0.00000 0.00000 20000 0.00005 0.00000 0.00000 20000 0.00005 0.00000 0.00000 30000 0.00003 0.00000 0.00000 30000 0.00003 0.00000 0.00000 40000 0.00003 0.00000 0.00000 40000 0.00003 0.00000 0.00000 50000 0.00002 0.00000 0.00000 50000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 60000 0.00002 0.00000 0.00000 70000 0.00001 0.00000 0.00000 70000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 80000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 90000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000 100000 0.00001 0.00000 0.00000

0.1 ≤100, ≥30000 ≤90, ≥30000 ≤90, ≥30000 ≤90, ≥30000 0.2 ≤20, ≥4000 ≤20, ≥4000 ≤10, ≥4000 ≤10, ≥4000 0.3 ≤10, ≥2000 ≤8, ≥2000 ≤8, ≥2000 ≤7, ≥2000 0.4 ≤7, ≥600 ≤4, ≥600 ≤4, ≥600 ≤4, ≥600 0.5 ≤5, ≥300 ≤3, ≥400 ≤3, ≥400 ≤2, ≥400 0.6 ≤3, ≥200 ≤2, ≥200 ≤2, ≥200 ≤1, ≥200 0.7 ≤3, ≥200 ≤1, ≥200 ≤1, ≥200 ≤1, ≥200 0.8 ≤2, ≥100 ≤1, ≥100 ≤1, ≥100 ≤1, ≥100 0.9 ≤2, ≥80 ≥ 80

Concluding Remarks

It may be noted that, for absolute values of θ not less than 1, For smaller absolute values of θ considered in the interval (-1, 1), it is found that when 'shifts' in θ from 0 are not very small (to the magnitude of 0.3 standard deviation units and above), As a prospective application of using SEE's, we consider a manufacturing process with a normally distributed quality characteristic. In this situation, suppose shifts in the process average of magnitudes exceeding 0.3 standard deviation units need to be correctly detected. This is required in various situations because these shifts, though not very large, will lead to the process not meeting the quality specifications. In such a scenario, the numerical study carried out in this research article suggests that drawing a sample of size above 2000 is recommended to get a closer estimate of the process average using the SEE's instead of the MLE.

A perusal of the 'Bias' columns in Tables 4

References

1. Bahadur RR. Hodges Super-Efficiency. In: Encyclopedia of Statistical Sciences. John-Wiley, 1983; 3:645-646.
2. Basu D. Unpublished dissertation. 1952.
3. Durairajan TM. Sub-score and Super-Efficient Estimator. In: Navin Chandra and Gopal, G., eds. Applications of Reliability Theory and Survival Analysis, Bonfring, 2012, 120-129.
4. Hodges JL. Unpublished. 1951.
5. Le Cam, L. On some asymptotic properties of maximum likelihood estimates and related Baye's estimates. University of California Publications in Statistics. 1953; 1:277-330.
6. Le Cam L. On the asymptotic theory of estimation and testing hypotheses. In: Neyman, J. ed. Proceedings of the Third Berkeley Symposium on Mathematics and Probability. University of California Press, Berkeley. 1956; 1:129-156.
7. Le Cam L. Locally asymptotically normal families of distributions. University of California Publications in Statistics. 1960; 3:37-98.
8. Le Cam L. Limits of experiments. In: Proceedings ofthe Sixth Berkeley Symposium on Mathematics and Probability. University of California Press, Berkeley. 1972; 1:245-261.
9. Sethuraman J. Are Super-Efficient Estimators Super-powerful?. Communications in Statistics -Theory and Methods. 2004; 33(9):2003-2013.
10. Sivasakthi K, Durairajan TM, Martin William L. Obtaining Super-Efficient Estimators for a Real-Valued Parameter: Delta Method. International Journal of Applied Mathematics and Statistical Sciences. 2017; 6(1):81-88.
11. Stein, C. Inadmissibility of the usual estimator for the mean of a normal distribution. In: Neyman, J. ed. Proceedings of the Third Berkeley Symposium on Mathematics and Probability. University of California Press, Berkeley. 1956; 1:197-206.