Lecture 9. A) Point Estimation (cont.)

From Significant Statistics
Jump to navigation Jump to search
.

Point Estimation (cont.)

Example: Uniform

Suppose [math]X_{i}\overset{iid}{\sim}U\left(0,\theta\right)[/math] where [math]\theta\gt 0[/math] is unknown.

The likelihood function equals [math]L\left(\left.\theta\right|x_{1}..x_{n}\right)=\Pi_{i=1}^{n}f\left(\left.x_{i}\right|\theta\right)=\Pi_{i=1}^{n}\frac{1}{\theta}1\left(0\leq x_{i}\leq\theta\right)[/math]

Since [math]x_{i}[/math]’s are draws from [math]U\left(0,\theta\right)[/math], the [math]0\leq x_{i}[/math] constraint will always be satisfied. However, since we are uncertain about the true value of [math]\theta[/math], the upper constraint may be binding.

This yields the following likelihood: [math]\Pi_{i=1}^{n}\frac{1}{\theta}1\left(0\leq x_{i}\leq\theta\right)=\Pi_{i=1}^{n}\frac{1}{\theta}1\left(x_{i}\leq\theta\right)=\frac{1}{\theta^{n}}1\left(x_{\left(n\right)}\leq\theta\right)[/math]

Notice that [math]L\left(\left.\cdot\right|x_{1}..x_{n}\right)[/math] is not differentiable at [math]\theta=x_{\left(n\right)}[/math]. We separate the problem:

  • [math]L\left(\left.\cdot\right|x_{1}..x_{n}\right)=0[/math] if [math]\theta\lt x_{\left(n\right)}[/math]; this reveals the impossibility that a value is generated above [math]\theta[/math].
  • [math]L\left(\left.\cdot\right|x_{1}..x_{n}\right)=\frac{1}{\theta^{n}}[/math] if [math]\theta\geq x_{\left(n\right)}[/math]; it is decreasing in [math]\theta[/math], so constraint is active, and [math]\widehat{\theta}_{ML}=x_{\left(n\right)}[/math].

Notice that the maximum likelihood estimator is different from the method of moments, [math]\widehat{\theta}_{ML}=x_{\left(n\right)}[/math] while [math]\widehat{\theta}_{MM}=2\overline{x}[/math].

Unlike the method of moments, we cannot obtain an estimator st [math]x_{i}\gt \widehat{\theta}_{ML}[/math].

However, as we will discuss later, there are some bad news.

The fact that we can never obtain [math]\widehat{\theta}_{ML}\gt \theta_{0}[/math], where [math]\theta_{0}[/math] is the true value of parameter [math]\theta[/math], means that the maximum likelihood estimator is likely to systematically underestimate the true parameter value.