Lecture 12. H) Equivalence Between LRT and Wald Tests

From Significant Statistics
Jump to navigation Jump to search

Equivalence Between LRT and Wald Tests

In this case, we expand the log-likelihood function around [math]\widehat{\theta}_{ML}[/math]:

[math]\begin{aligned} l\left(\theta\right) & \simeq l\left(\widehat{\theta}_{ML}\right)+\underset{=0}{\underbrace{l^{'}\left(\widehat{\theta}_{ML}\right)}}\left(\theta-\widehat{\theta}_{ML}\right)+\frac{l^{''}\left(\widehat{\theta}_{ML}\right)}{2}\left(\theta-\widehat{\theta}_{ML}\right)^{2}\\ & =l\left(\widehat{\theta}_{ML}\right)+\frac{l^{''}\left(\widehat{\theta}_{ML}\right)}{2}\left(\theta-\widehat{\theta}_{ML}\right)^{2}\end{aligned}[/math]

Now, plugging this result evaluated at [math]\theta_{0}[/math] into the LRT yields:

[math]\begin{aligned} 2\left[l\left(\widehat{\theta}_{ML}\right)-l\left(\theta_{0}\right)\right] & \simeq2\left[l\left(\widehat{\theta}_{ML}\right)-\left(l\left(\widehat{\theta}_{ML}\right)+\frac{l^{''}\left(\widehat{\theta}_{ML}\right)}{2}\left(\theta_{0}-\widehat{\theta}_{ML}\right)^{2}\right)\right]\\ & =-l^{''}\left(\widehat{\theta}_{ML}\right)\left(\theta_{0}-\widehat{\theta}_{ML}\right)^{2}\\ & =\frac{\left(\theta_{0}-\widehat{\theta}_{ML}\right)^{2}}{-l^{''}\left(\widehat{\theta}_{ML}\right)^{-1}}\end{aligned}[/math]

which yields the Wald test.

Notice that the log-likelihood of the Normal distribution is quadratic in [math]\mu[/math], such that these 3 procedures produce exactly the same test for [math]\mu[/math] (when [math]\sigma^{2}[/math] is known), because we have used quadratic approximations to the log-likelihood function.

Finally, when the null hypothesis is composite, it is usually possible to construct an LM and Wald test.