Lecture 2. A) Random Variables (cont.)

From Significant Statistics
Jump to navigation Jump to search

Correspondence Theorem

Let [math]P_{X}\left(\cdot\right)[/math] and [math]P_{Y}\left(\cdot\right)[/math] be probability functions, defined on [math]\mathcal{B}\left(\mathbf{R}\right)[/math] and let [math]F_{X}\left(\cdot\right)[/math] and [math]F_{Y}\left(\cdot\right)[/math] be associated cdfs. Then,

[math]P_{X}\left(\cdot\right)=P_{Y}\left(\cdot\right)[/math] iff [math]F_{X}\left(\cdot\right)=F_{Y}\left(\cdot\right).[/math]

The correspondence theorem assures us that we can restrict ourselves to cdfs. Relying on these won’t restrict us in any way, when compared to using probability functions.


Function [math]F:\mathbf{R}\rightarrow\left[0,1\right][/math] is a cdf if it satisfies the following conditions:

  • [math]\lim_{x\rightarrow-\infty}F\left(x\right)=0.[/math]
  • [math]\lim_{x\rightarrow+\infty}F\left(x\right)=1.[/math]
  • [math]F\left(\cdot\right)[/math] is non-decreasing.
  • [math]F\left(\cdot\right)[/math] is right-continuous (this can be shown by using probability functions of intervals).

Nature of RVs

We now define the natures of random variables:

  • Random variable [math]X[/math] is discrete if

[math]\exists f_{X}:\mathbf{R}\rightarrow\left[0,1\right][/math] s.t. [math]F_{X}\left(x\right)=\sum_{t\leq x}f_{X}\left(t\right),x\in\mathbf{R}.[/math]

Function [math]f_{X}[/math] is called the probability mass function (pmf).

  • Random variable [math]X[/math] is continuous if

[math]\exists f_{X}:\mathbf{R}\rightarrow\mathbf{R}_{+}[/math] s.t. [math]F_{X}\left(x\right)=\int_{-\infty}^{x}f_{X}\left(t\right)dt,x\in\mathbf{R}.[/math]

Any such [math]f_{X}[/math] is called a probability density function (pdf). Notice that unlike pmfs, multiple pdfs are consistent with a given cdf. This occurs as long as the pdfs differ only on a set of (probability) measure-zero events.

Another interesting remark is that the probability of any specific value of a continuous variable is zero, i.e., [math]P\left(\left\{ x\right\} \right)=0,\forall x\in\mathbf{R}[/math].


Coin tossing

[math]F_{X}\left(x\right)=\begin{cases} 0, & x\lt 0\\ \frac{1}{2}, & 0\leq x\lt 1\\ 1, & x\geq1 \end{cases}[/math]

In this case, [math]X[/math] is discrete and [math]F_{X}[/math] is a step function (this always occurs for discrete r.v.s).

The probability mass function is equal to [math]f_{X}\left(x\right)=\begin{cases} \frac{1}{2}, & x\in\left\{ 0,1\right\} \\ 0, & otherwise \end{cases}[/math].

Uniform distribution on (0,1)

[math]F_{X}\left(x\right)=\begin{cases} 0, & x\lt 0\\ x, & 0\leq x\lt 1\\ 1, & x\geq1 \end{cases}[/math] where [math]X[/math] is continuous.

Moreover, both [math]f_{X}\left(x\right)=\begin{cases} 1, & x\in\left[0,1\right]\\ 0, & otherwise \end{cases}[/math] and [math]f_{X}\left(x\right)=\begin{cases} 1, & x\in\left(0,1\right)\\ 0, & otherwise \end{cases}[/math] are consistent pdfs.

Normal distribution

A r.v. [math]X[/math] has a standard normal distribution, [math]X\sim N\left(0,1\right)[/math], if it is continuous with pdf


PMFs and PDFs

Notice that pmfs and, in a sense pdfs, ‘add up’ to one. There is a theorem that states the result applies in both directions.

For the pmf,

[math]f:\mathbf{R}\rightarrow\left[0,1\right][/math] is the pmf of a discrete r.v. iff [math]\sum_{x\in\mathbf{R}}f\left(x\right)=1.[/math]

And for the pdf,

[math]f:\mathbf{R}\rightarrow\mathbf{R}_{+}[/math] is the pdf of a continuous r.v. iff [math]\int_{-\infty}^{\infty}f\left(x\right)dx=1.[/math]


It’s clear from the examples above, that one can specify the distribution of a random variable by specifying its distribution function, or its probability mass/density function. Sometimes, however, it is advantageous to specify the distribution of a random variable by a transformation. For example, suppose [math]Y[/math] is defined as a random variable that follows the distribution of [math]X^{2}[/math], where [math]X\sim N\left(0,1\right)[/math]. This takes us to discussing transformations of random variables. But first, we'll see a very useful result.