Lecture 13. G) Interval Estimation/Confidence Intervals
Interval Estimation/Confidence Intervals
In interval estimation, we would like to isolate plausible values of some parameter [math]\theta[/math].
Remember that in point estimation, we have attempted to find the “most plausible” value of [math]\theta[/math]. In hypothesis testing, we assigned [math]\theta[/math] to one of two subsets of [math]\Theta[/math]. In interval estimation, we produce an interval that is likely to contain the true parameter [math]\theta[/math].
Let [math]\Theta\subset\mathbb{R}[/math]. An interval estimator of [math]\theta[/math] is a random interval [math]\left[L\left(X_{1}..X_{n}\right),U\left(X_{1}..X_{n}\right)\right][/math] where [math]L\left(\cdot\right)[/math] and [math]U\left(\cdot\right)[/math] are statistics. We will use interval estimators and confidence intervals as synonymous. (This works well as long as we don't nitpick.)
- The coverage probability of the interval is the function (of [math]\theta[/math]): [math]P_{\theta}\left(L\left(X_{1}..X_{n}\right)\leq\theta\leq U\left(X_{1}..X_{n}\right)\right)[/math] where the subscript in [math]P_{\theta}[/math] stresses the fact that [math]\theta[/math] is a scalar and not a random variable.
- The confidence coefficient is [math]\text{inf}_{\theta\in\Theta}P_{\theta}\left(L\left(X_{1}..X_{n}\right)\leq\theta\leq U\left(X_{1}..X_{n}\right)\right)[/math] i.e., for all values of [math]\theta\in\Theta[/math], the confidence coefficient is the minimum of the coverage probabilities.
Example: Normal
Let [math]X_{i}\overset{iid}{\sim}N\left(\mu,\sigma^{2}\right)[/math], where [math]\sigma^{2}[/math] is known. The interval [math]\left[\overline{X}-1.96\frac{\sigma}{\sqrt{n}},\overline{X}+1.96\frac{\sigma}{\sqrt{n}}\right][/math] is a 95% confidence interval (i.e., it has a confidence coefficient of 95%) of [math]\mu[/math]. To see this, note that
[math]\begin{aligned} & P_{\mu}\left(\overline{X}-1.96\frac{\sigma}{\sqrt{n}}\leq\mu\leq\overline{X}+1.96\frac{\sigma}{\sqrt{n}}\right)\\ = & P_{\mu}\left(-1.96\frac{\sigma}{\sqrt{n}}\leq\mu-\overline{X}\leq1.96\frac{\sigma}{\sqrt{n}}\right)\\ = & P_{\mu}\left(-1.96\frac{\sigma}{\sqrt{n}}\leq\overline{X}-\mu\leq1.96\frac{\sigma}{\sqrt{n}}\right)\\ = & P_{\mu}\left(-1.96\leq\underset{\sim N\left(0,1\right)}{\underbrace{\frac{\sqrt{n}\left(\overline{X}-\mu\right)}{\sigma}}}\leq1.96\right)\simeq0.95.\end{aligned}[/math]
Notice in this case that the coverage probability is equal to [math]0.95[/math] at all values of [math]\mu[/math], such that the confidence coefficient does not depend on [math]\mu.[/math]
A common way to obtain confidence intervals is to 'invert' tests.
For example, consider the two-sided [math]Z[/math]-test of [math]H_{0}:\mu=\mu_{0}[/math] vs. [math]H_{1}:\mu\neq\mu_{0}[/math], that rejects [math]H_{0}[/math] iff [math]\overline{X}-1.96\frac{\sigma}{\sqrt{n}}\gt \mu_{0}\text{ or }\overline{X}+1.96\frac{\sigma}{\sqrt{n}}\lt \mu_{0}.[/math]
Notice that the test accepts [math]H_{0}[/math] iff [math]\overline{X}-1.96\frac{\sigma}{\sqrt{n}}\leq\mu_{0}\leq\overline{X}+1.96\frac{\sigma}{\sqrt{n}}.[/math]
The confidence interval [math]\left[\overline{X}-1.96\frac{\sigma}{\sqrt{n}},\overline{X}+1.96\frac{\sigma}{\sqrt{n}}\right][/math] consists of those values of [math]\mu_{0}[/math] for which the two-sided [math]z[/math]-test does not reject [math]H_{0}:\mu=\mu_{0}[/math]. In other words, there is a duality between 5% tests and 95% confidence intervals.
In general, we create a 95% confidence set for [math]\theta\in\Theta[/math] by including those values of [math]\theta_{0}[/math] for which a 5% test of [math]H_{0}:\theta=\theta_{0}[/math] does not reject.
Optimality
A confidence set is optimal iff it is obtained by inverting an optimal test. Because an optimal test minimizes the probability of type 2 errors (i.e., accepting the null hypothesis when the alternative hypothesis was correct), the corresponding optimal interval is optimal in the sense that it minimizes the likelihood of including values of [math]\mu\neq\mu_{0}[/math] in the confidence interval.
