Lecture 7. C) Order Statistics
Order Statistics
Let [math]X_{1}..X_{n}[/math] be a random sample. The order statistics are the sample values placed in ascending order, i.e.,
[math]X_{\left(1\right)}=\min_{i\leq n}X_{i}\leq X_{\left(2\right)}\leq...\leq X_{\left(n\right)}=\max_{i\leq n}X_{i}[/math]
This is a maybe unexpected, but often useful statistic. We can ask what the is distribution of the maximum of a random sample.
For example, if we drew many sets of 30 draws each of [math]X\sim N\left(0,1\right)[/math], what would be the distribution of the maximum?
Distribution of the Maximum
The distribution of the maximum of a random sample with cdf [math]F\left(\cdot\right)[/math] equals
[math]F_{X_{\left(n\right)}}\left(x\right)=P\left(X_{\left(n\right)}\leq x\right)=P\left(X_{1}\leq x,X_{2}\leq x,...,X_{n}\leq x\right)=P\left(X_{1}\leq x\right)P\left(X_{2}\leq x\right)...P\left(X_{n}\leq x\right)=F\left(x\right)^{n}[/math].
The distribution for the lowest order statistic can also be calculated via a similar method.
Distribution of Order Statistics
In general, the distribution of the k-th order statistic is given by
[math]F_{X_{\left(r\right)}}\left(x\right)=P\left(X_{\left(r\right)}\leq x\right)=\sum_{j=r}^{n}\left(\begin{array}{c} n\\ j \end{array}\right)F\left(x\right)^{j}\left(1-F\left(x\right)\right)^{n-j}[/math]
where the binomial structure is apparent.
For each value of [math]j[/math], starting at [math]r[/math], we sum the probability of observing [math]j[/math] values below [math]x[/math] and [math]n-j[/math] values above.
For example, consider the case where [math]n=30[/math]. Then, [math]P\left(X_{\left(r\right)}\leq x\right)=P\left(X_{\left(r\right)}\leq x\wedge X_{\left(r+1\right)}\gt x\right)+\left(X_{\left(r+1\right)}\leq x\wedge X_{\left(r+2\right)}\gt x\right)+...[/math]: The sum over the binomials is simply the sum of the probabilities of the cases that satisfy [math]X_{\left(r\right)}\leq x[/math].
