Lecture 12. I) Optimal Tests

From Significant Statistics
Jump to navigation Jump to search

Optimal Tests

We first provide a couple of definitions in regards to tests:

  • A test has level [math]\alpha[/math] if [math]\beta\left(\theta\right)\leq\alpha[/math][math]\forall\theta\in\theta_{0}[/math].
  • The size of a test is [math]\sup_{\theta\in\Theta_{0}}\,\beta\left(\theta\right)[/math].

Notice that a test has multiple levels. A 5% level test has less than a 5% probability of a type 1 error. However, this test is also a level 10% test, level 15%, etc.

A test with size [math]5\%[/math] means that the highest probability (among the potential values of the true parameter [math]\theta[/math]) is 5%.

Neyman Approach

In order to define test optimality, we will follow the Neyman approach. We will select a class [math]C[/math] of tests with level [math]\alpha[/math] (e.g., [math]\alpha=0.05[/math]). Then, we minimize the probability of type 2 errors (for all possible values of [math]\theta\in\Theta_{1}[/math]), with the constraint that the level of the test is fixed at [math]5\%[/math].

Test Optimality (UMP)

First, let the testing problem be

[math]H_{0}:\theta\in\Theta_{0}\,vs.\,H_{1}:\theta\in\Theta_{1}[/math]

and let [math]C[/math] be a collection of tests.

A test in [math]C[/math] with power function [math]\beta\left(\cdot\right)[/math] is a uniformly most powerful (UMP) [math]C[/math] test if

[math]\beta\left(\theta\right)\geq\beta^{*}\left(\theta\right),\,\forall\,\theta\in\Theta_{1}[/math] for every [math]\beta^{*}[/math] corresponding to a test in [math]C[/math].

The UMP test has the lowest type 2 error probability among tests with level [math]\alpha[/math].

Finding an UMP by hand is challenging. This is where the Neyman-Pearson lemma comes in.