Research article · 2017 · Full text
International Journal of Statistics and Applied MathematicsMaximum probability estimation for an autoregressive process
Volume 2(6) · Pages 122–123
Research areasFault Detection and Control Systems
Abstract
Author-writtenWe observe Xi, …., Xn, where Xi = θi Xi – 2 x Yi, ---------- (1) Where Xθ is defined as zero, and Yi,…., Yn are unobservable random variables, independent, each normal with mean zero and variance θ2. θ1 and θ2 are both unknown and are to be estimated by using Maximum Probability Estimation. It is shown that the maximum likelihood estimators of the parameters have certain optimal properties.
Summary
Source-checkedPlain-language reading of the full paper, prepared with AI and checked against the text. Each point links to the authors' own words.
Why it matters
- Maximum probability estimators were developed to provide a theoretical explanation for the effectiveness of maximum likelihood estimators in various scenarios. View evidence ↗
- Maximum probability estimators are defined by their ability to provide the highest asymptotic probability that the difference between the estimator and the true parameter falls within a shrinking region as the sample size increases. View evidence ↗
The question
- The research aims to apply maximum probability estimation to determine the unknown parameters θ1 and θ2 in an autoregressive process where observations Xi depend on the previous value Xi-1 and an unobservable normal random variable Yi. View evidence ↗
What they did
- The study utilizes the joint density function of the observed sequence X1 through Xn to derive estimators. View evidence ↗
- The authors employ maximum likelihood estimators for θ1 (based on the sum of products of consecutive observations) and θ2 (based on the average squared difference between observations and their predicted values). View evidence ↗
What they found
- When the absolute value of θ1 is less than 1, the maximum likelihood estimators for θ1 and θ2 follow an asymptotic bivariate normal distribution with zero means and zero covariance. View evidence ↗
- For |θ1| < 1, the maximum likelihood estimators share the same asymptotic properties as maximum probability estimators for any convex region symmetric about the origin. View evidence ↗
- The estimation of θ1 is independent of whether the value of θ2 is known or unknown. View evidence ↗
- When θ2 is known to be one, the maximum likelihood estimator for θ1 is also the maximum probability estimator for symmetric regions and is arbitrarily close to a Bayes estimator. View evidence ↗
- If the absolute value of θ1 is greater than 1, the asymptotic distribution of the estimator for θ2 is identical to the distribution that would be obtained if θ1 were known. View evidence ↗
- The estimator for θ2 (when θ1 is known) follows an asymptotic standard normal distribution, derived from a chi-square distribution with n degrees of freedom. View evidence ↗
- In the case where |θ1| = 1, the difference between the estimator and the parameter is of order Op(1/n). View evidence ↗
What it means
- The authors conclude that maximum likelihood estimators for the parameters of this autoregressive process possess optimal properties. View evidence ↗
Visual summary
1 visualsIllustrations explain the paper; they do not show study data.
Autoregressive Process Flow
AI-generated conceptual illustration (A sequence of mathematical data points represented as spheres in a 3D coordinate system). It explains the idea and does not show study data.
View source evidence ↗Introduction
Maximum probability estimators were decolorized in Weiss and Wolfowitz (2007, 2008, 2009, 2010) [3][4][5][6] . They represented an attempt to explain why maximum likelihood estimators work so well in many cases. The brief description of this asymptotic. A brief description of this asymptotic theory follows. There is an index n, which takes positive integral values, and we examine what happens as n increases. For each n, X(n) is the random vectors to be observed. The joint density function for the components of X(n) is fn(X (n)I θi, …, θm), where θ1, ….., θm are unknown parameters to be (θ1, ….., θm), Rn (θ1, …., θm) is a bounded measurable subset of mdimensional space, containing the m -dimensional origin (θ1, … θ)i and as n increases, Rn (θ1,…. θm) shrinks towards this origin. For any given value, D1, …., Dm let Rn * (D1, …., Dm) denote the set of vectors(θ1, …, θm) such that the vector (D1 -θ1, …., Dmθm) is in Rn (θ1, …., θm). let θ1(n) ….., θm(n) denoteteh values of D1, …., Dm which maximize the integral ∫..∫ fn 1 X (n)i θ1, …., θm) dθ1, …, dθm ----------
(2) Rn (D1,... Dm) 𝜃 ̂1 (n) …, 𝜃 ̂m(n) are called "Maximum probability estimators with respect to Rn(θ1, ….., θm)" because if θ1, …, θm can be estimated with increasing accuracy as n increases, then among all estimators satisfying a reasonable regularity condition, (𝜃 ̂1(n), …., 𝜃 ̂m(n)) gives the highest asymptotic probability that the vector (estimator of θi -𝜃 ̂1) …. estimator of θm -𝜃 ̂m) falls in Rn(θ1, … θm). In Weiss (2004) [2] , small-sample properties of maximum probability estimators were investigated. Now, we apply the theory of maximum probability estimators to the following estimation problem. Y1, …., Yn are independent and identically distributed, each normal with mean zero and unknown positive variance θ2. We do not observe Y1, …. Yn, what we observe are Xi, …., Xn, where Xi = θ1 x i -1 + Yi ---(3) with Xo defined as zero. θ1 is an unknown parameter. The problem is to estimate θ1 and θ2.
The joint density function for
) and the maximum likelihood estimators are ;
In Weiss (2004) [2] and Weiss and Wolfowitz (1968), this process was studied for the ease where θ2 was known to be equal to one, and so only θ1 had to be estimated.
The Case / θ1/ < 1
If / θ1/<1, it is easily verified that the conditions of Weiss (2003) [1] are satisfied, and therefore the maximum likelihood estimators have the following properties. n 1/2 (θ1 -𝜃 ̂1) and n 1/2 (θ2 -𝜃 ̂2) have asymptotically a bivariate normal distribution with zero means, respective variances 1 -θ1 2 , 2 θ2 2 , and covariance zero. Furthermore, 𝜃 ̂1, 𝜃 ̂2 have the same asymptotic properties as maximum probability estimators with respect to any convex region symmetric about the origin (0, 0)
The Estimation of θ1
Known θ2 is of no advantage in estimating θ1. That is, a statistician who does not know θ2 can estimate θ1 as well as statistician who know the value of θ2. To see this, imagine that unknown to us, each Y1 is multiplied by a nonzero value c. this replaces θ2 by C 2 θ2, and multiplies each X1 by C, but does not change either 𝜃 ̂1 or θ1. The fact that knowing θ2 is of no relevance for estimating θ1 explains the form of asymptotic convenience matrix in section 2.
In Weiss (2004) [2] , the problem of estimating θ1 when θ2 is known to be one and θ1 is completely unknown as studied. In this case, the maximum likelihood estimator 𝜃 ̂1 is also the maximum probability estimator with respect to any region Rn(θ1) symmetric around zero, and it was shown that 𝜃 ̂1 is uniformly arbitrarily close to a Bayes estimator. For each n > 2. From the discussion above, it follows that these desirable properties of 𝜃 ̂1 also hold if θ2 is unknown.
Asymtotic Properties Of 𝜽 ̂2 When / θ1/ >, 1
In this section, we show that if / θ1/>,1, the asymptotic distribution of θ2 is the same as that of the maximum likelihood estimator of θ2 which would be available to somebody who knew the value of θ1. Denote by θ2 * the maximum likelihood estimator of θ2 when θ1 is known.
Since nθ2 * /θ2 has a chi-square distribution with n degrees of freedom, it follows that (n/2θ2 2 ) 1/2 (θ2 x -θ2) is asymptotically standard normal.
Let R denote ∑ n i = 1 Xi -1 Y1 and let S denote
The following relations are easily verified: -------(9) U We will show that if /θ1/≥1, then (𝜃 ̂1 -θ1) 2 S /n 1/2 converses to as n increases: considering (9) and the asymptotic distribution of θ2 * , this will prove that the asymptotic distribution of ( n /2 θ2 2 ) 1/2 (𝜃 ̂2 -θ2) is standard normal.
Discussion
First, we discuss the case where /θ1/>1. Since X1 = ∑ n i -1 θ1 i -1 Yi for i > 1. X1 has a normal distribution with mean zero and variance [θ2 2 / (1 -θ1
Thus, R is Op (/θ1/ n ). Xn is Op (/θ1/ 2n ), and ∑ n 1 =1 Yi 2 is Op (/θ1/ *n ). Then (𝜃 ̂1 -θ1) 2 S/n 1/2 converses to zero in probability as n increases. Now we look at the case where / θ1/ = 1. We carry out the details for the case θ1 = -1 being essentially the same. In this case R has mean θ2n (n -1)/2. In computing the variance of S, we use the fact that if j > i, then Xi = Xi + (Yi + 1 Y ……. + Y1), and (Yi + 1 Y ……. + Y1) is independent of straightforward calculation then shows that the variance of S is θ2 2 n 4 /12 + θ2 2 O(n 3 ). We can write R as θ2/n (n -1) 12 + 12R 1 where R i is Op (1), and we can write S as θ2n (n -1) 12 + θ2n 2 S 1 / (12) 1/2 where S 1 is Op (1). It follows directly that 𝜃 ̂i -θ1 = R/S is Op ( 1 /n), and then (𝜃 ̂i -θ1) 2 S/n 1/2 = Op (n 1/2 ). This completes the demonstration.
References
6 in the original paper- Weiss. Asymptotic properties of maximum likelihood estimators in some nonstandard cases. Part II, Journal of American statistical Association. 2003; 68:728-730.
- Weiss. Small-sample properties of maximum probability estimators, stochastic process and their Applications. 2004; 14:367-277.
- Weiss, Wolfowitz. Maximum probability estimators. American Institute of Mathematical Statistics. 2007; 19, 193-206.
- Weiss, Wolfowitz J. Generalized maximum likelihood estimators. Journal of American Statistical Association. 2008; 88:68-97.
- Weiss, Wolfowitz J. Maximum probability estimators and asymptotic probability estimators and asymptotic sufficiency. American institute of Mathematical statistics 2009; 22:225-244.
- Weiss, Wolfowitz J. Maximum probability Estimators and Related topics. Springer. Berlin, 2010.