Lecture 2. C) Transformations of Random Variables
Transformations of random variables
Suppose [math]Y=g\left(X\right)[/math], where [math]g:\mathbf{R}\rightarrow\mathbf{R}[/math] is a function and [math]X[/math] is an r.v. with cdf [math]F_{X}[/math].
Clearly, [math]Y[/math] is also a random variable. Its induced probability function is equal to [math]P_{Y}\left(\cdot\right)=P_{X}\circ g^{-1}[/math]. When [math]X[/math] is discrete, it is usually simple to obtain the distribution of [math]Y[/math]. This becomes more complicated in the continuous case.
We consider the cases of strictly monotone transformations here. When transformations are not strictly monotone, the same procedure applies in a piecewise fashion (i.e., one needs to apply it repeatedly to different monotone sections of the transformation).
Affine Transformations: CDF
- Suppose [math]Y=g\left(X\right)=aX+b,a\gt 0,b\in\mathbf{R}[/math].
In order to deduce [math]F_{Y}[/math], we use the probability functions of [math]X[/math] and [math]Y[/math]. Notice first that [math]F_{Y}\left(y\right)=P\left(Y\leq y\right)[/math]. This probability statement can be used to relate the cdf of [math]Y[/math] to the cdf of [math]X[/math]:
[math]P\left(Y\leq y\right)=P\left(aX+b\leq y\right)=P\left(X\leq\frac{y-b}{a}\right)=F_{X}\left(\frac{y-b}{a}\right).[/math]
This is a very useful result: we have related the cdf of a transformed r.v. [math]Y[/math] to the cdf of the transformed variable [math]X[/math]. We have learned that the distribution of [math]Y[/math] is given by the distribution of [math]X[/math], evaluated at a transformed value of the function's argument.
- Now, suppose [math]Y=aX+b[/math] where [math]a\lt 0[/math]. In this case, we obtain
[math]F_{Y}\left(y\right)=P\left(Y\leq y\right)=P\left(aX+b\leq y\right)=P\left(X\geq\frac{y-b}{a}\right)=1-P\left(X\leq\frac{y-b}{a}\right)=1-F_{X}\left(\frac{y-b}{a}\right)[/math].
Affine Transformations: PDF
- Let [math]a\gt 0[/math] and [math]Y=aX+b[/math].
We know [math]F_{Y}\left(y\right)=F_{X}\left(\frac{y-b}{a}\right)[/math], and that [math]f_{Y}\left(y\right)=\frac{d}{dy}F_{Y}\left(y\right).[/math]
By applying Leibniz rule, we obtain
[math]f_{Y}\left(y\right)=\frac{d}{dy}F_{X}\left(\frac{y-b}{a}\right)=f_{X}\left(\frac{y-b}{a}\right)\frac{d}{dy}\frac{y-b}{a}=f_{X}\left(\frac{y-b}{a}\right)\frac{1}{a}[/math].
- If, on the other hand, [math]a\lt 0[/math], we would have [math]F_{Y}\left(y\right)=1-F_{X}\left(\frac{y-b}{a}\right)[/math], and applying Leibniz rule yields [math]f_{Y}\left(y\right)=-f_{X}\left(\frac{y-b}{a}\right)\frac{1}{a}.[/math]
We can write down both of these cases simultaneously, as
[math]f_{Y}\left(y\right)=f_{X}\left(\frac{y-b}{a}\right)\left|\frac{1}{a}\right|[/math], when [math]Y=aX+b[/math] and [math]a\neq 0[/math].
In general, as long as the transformation [math]Y=g\left(X\right)[/math] is monotonic, then
[math]f_{Y}\left(y\right)=f_{X}\left(g^{-1}\left(y\right)\right)\left|\frac{d}{dy}g^{-1}\left(y\right)\right|.[/math]
When it is not, then one can simply apply the formula separately for each monotonic region.
Also, notice that the role of [math]g^{-1}\left(y\right)[/math] is to ensure that the result is expressed as a function of the argument of interest, [math]y[/math], rather than [math]x[/math].
There also exists a formula for transformations of multiple random variables. In this case, rather than multiplying the pdf by a single derivative, one uses the absolute value of the determinant of the Jacobian matrix of the transformations.
