Lecture 12. B) Likelihood-Ratio Test
Likelihood Ratio Test
Let [math]X_{1}..X_{n}[/math] be a random sample with pmf/pdf [math]f\left(\left.\cdot\right|\theta\right)[/math], where [math]\theta\in\Theta\subseteq\mathbb{R}[/math], and consider the test
[math]H_{0}:\,\theta=\theta_{0}\,\text{vs. }\theta\neq\theta_{0}[/math]
Recall that the log-likelihood function is [math]l\left(\left.\theta\right|x_{1}..x_{n}\right)=\sum_{i=1}^{n}\text{log}\left(f\left(\left.x_{i}\right|\theta\right)\right)[/math]
The Likelihood Ratio test (LRT) statistic is
[math]T_{LR}\left(X_{1}..X_{n}\right)=2\left[l\left(\left.\widehat{\theta}_{ML}\right|X_{1}..X_{n}\right)-l\left(\left.\theta_{0}\right|X_{1}..X_{n}\right)\right][/math]
In order to calculate this statistic, we simply take the difference of the log-likelihoods evaluated at the ML estimator and the value of the null hypothesis (assuming [math]H_{0}[/math] is a simple hypothesis).
The result is the test statistic that will be compared with the cutoff point (aka, the critical value).
A Few Important Notes
- We have just produced a test, which will yield a value whenever we apply it to data. However, we were not told anything about its distribution. We will discuss this later.
- If [math]H_{0}[/math] is composite, then we calculate [math]T_{LR}\left(X_{1}..X_{n}\right)=2\left[l\left(\left.\widehat{\theta}_{ML}\right|X_{1}..X_{n}\right)-\text{sup}_{\theta_{0}\in\Theta_{0}}l\left(\left.\theta_{0}\right|X_{1}..X_{n}\right)\right][/math] instead.
- The presence of the [math]2[/math] will be explained later. Clearly, it does not affect the test statistic, since removing it will simply scale the critical value by one half.
- The LRT is motivated by the Neyman-Pearson Lemma, which we will discuss later.
- The intuition for the test statistic is as follows. Suppose [math]T_{LR}[/math] is very high. This means that the likelihood of the sample at the most likely value of [math]\widehat{\theta}_{ML}[/math], is very far from value of the null hypothesis. If the difference is very high, [math]\theta[/math] is unlikely to equal [math]\theta_{0}[/math], and we will reject the null hypothesis.
