Lecture 12. I) Optimal Tests
Optimal Tests
We first provide a couple of definitions in regards to tests:
- A test has level [math]\alpha[/math] if [math]\beta\left(\theta\right)\leq\alpha[/math], [math]\forall\theta\in\theta_{0}[/math].
- The size of a test is [math]\sup_{\theta\in\Theta_{0}}\,\beta\left(\theta\right)[/math].
Notice that a test has multiple levels. A 5% level test has less than a 5% probability of a type 1 error. However, this test is also a level 10% test, level 15%, etc.
A test with size [math]5\%[/math] means that the highest probability (among the potential values of the true parameter [math]\theta[/math]) is 5%.
Neyman Approach
In order to define test optimality, we will follow the Neyman approach. We will select a class [math]C[/math] of tests with level [math]\alpha[/math] (e.g., [math]\alpha=0.05[/math]). Then, we minimize the probability of type 2 errors (for all possible values of [math]\theta\in\Theta_{1}[/math]), with the constraint that the level of the test is fixed at [math]5\%[/math].
Test Optimality (UMP)
First, let the testing problem be
[math]H_{0}:\theta\in\Theta_{0}\,vs.\,H_{1}:\theta\in\Theta_{1}[/math]
and let [math]C[/math] be a collection of tests.
A test in [math]C[/math] with power function [math]\beta\left(\cdot\right)[/math] is a uniformly most powerful (UMP) [math]C[/math] test if
[math]\beta\left(\theta\right)\geq\beta^{*}\left(\theta\right),\,\forall\,\theta\in\Theta_{1}[/math] for every [math]\beta^{*}[/math] corresponding to a test in [math]C[/math].
The UMP test has the lowest type 2 error probability among tests with level [math]\alpha[/math].
Finding an UMP by hand is challenging. This is where the Neyman-Pearson lemma comes in.
