We now introduce normal random variables, which are also often called Gaussian random variables.
Normal random variables are perhaps the most important ones in probability theory.
They play a key role in the theory of the subject, as we will see later in this class in the context of the central limit theorem.
They’re also prevalent in applications for two reasons.
They have some nice analytical properties, and they’re are also the most common model of random noise.
In general, they are a good model of noise or randomness whenever that noise is due to the addition of many small independent noise terms, and this is a very common situation in the real world.
We define normal random variables by specifying their PDFs, and we start with the simplest case of the so-called standard normal.
The standard normal is indicated with this shorthand notation, and we will see shortly why this notation is being used.
It is defined in terms of a PDF.
This PDF is defined for all values of x.
x can be any real number.
So this random variable can take values anywhere on the real line.
And the formula for the PDF is this one.
Let us try to understand this formula.
So we have the exponential of negative x squared over 2.
Now, if we are to plot the x squared over 2 function, it has a shape of this form, and it is centered at zero.
But then we take the negative exponential of this function.
Now, when you take the negative exponential, whenever this thing is big, the negative exponential is going to be small.
So the negative exponential would be equal to 1 when x is equal to 0.
But then as x increases, because x squared also increases, the negative exponential will fall off.
And so we obtain a shape of this kind, and symmetrically on the other side as well.
And finally, there is this constant.
Where [is] this constant coming from? Well there’s a nice and not completely straightforward calculus exercise that tells us that the integral from minus infinity to plus infinity of e to the negative x squared over 2, dx, is equal to the square root of 2 pi.
Now, we need a PDF to integrates to 1.
And so for this to happen, this is the constant that we need to put in front of this expression so that the integral becomes 1, and that explains the presence of this particular constant.
What is the mean of this random variable? Well, x squared is symmetric around 0, and for this reason, the PDF itself is symmetric around 0.
And therefore, by symmetry, the mean has to be equal to 0.
And that explains this entry here.
How about the variance? Well, to calculate the variance, you need to solve a calculus problem again.
You need to integrate by parts.
And after you carry out the calculation, then you find that the variance is equal to 1, and that explains this entry here in the notation that we have been using.
Let us now define general normal random variables.
General normal random variables are once more specified in terms of the corresponding PDF, but this PDF is a little more complicated, and it involves two parameters– mu and sigma squared, where sigma is a given positive parameter.
Once more, it will have a bell shape, but this bell is no longer symmetric around 0, and there is some control over the width of it.
Let us understand the form of this PDF by focusing first on the exponent, exactly as we did for the standard normal case.
The exponent is a quadratic, and that quadratic is centered at x equal to mu.
So it vanishes when x is equal to mu, and becomes positive elsewhere.
Then we take the negative exponential of this quadratic, and we obtain a function which is largest at x equal to mu, and falls off as we go further away from mu.
What is the mean of this random variable? Since this term is symmetric around mu, the PDF is also symmetric around mu, and therefore, the mean is also equal to mu.
How about the variance? It turns out– and this is a calculus exercise that we will omit– that the variance of this PDF is equal to sigma squared.
And this explains this notation here.
We’re dealing with a normal that has a mean of mu and a variance of sigma squared.
To get a little bit of understanding of the role of sigma in the form of this PDF, let us consider the case where sigma is small, and see how the picture is going to change.
When sigma is small, and we plot the quadratic, sigma being small means that this quadratic becomes larger, so it rises faster, so we get a narrower quadratic.
And in that case, the negative exponential is going to fall off much faster.
So when sigma is small, the PDF that we get is a narrower PDF, and that reflects itself into the property that the variance will also be smaller.
An important property of normal random variables is that they behave very nicely when you form linear functions of them.
And this is one of the reasons why they’re analytically tractable and analytically very convenient.
Here is what I mean.
Let us start with a normal random variable with a given mean and variance, and let us form a linear function of that random variable.
What is the mean of Y? Well, we know what it is.
We have a linear function of a random variable.
The mean is going to be a times the expected value of X, which is mu plus b.
What is the variance of Y? We know what is the variance of a linear function of a random variable.
It is a squared times the variance of X, which in our case is sigma squared.
So there’s nothing new so far, but there is an additional important fact.
The random variable Y, of course, has the mean and variance that we know it should have, but there is an additional fact– namely, that Y is a normal random variable.
So normality is preserved when we form linear functions.
There’s one special case that’s we need to pay some attention to.
Suppose that a is equal to 0.
In this case, the random variable Y is just equal to b.
It’s a constant random variable.
It does not have a PDF.
It is a degenerate discrete random variable.
So could this fact be correct that Y is also normal? Well, we’ll adopt this as [a] convention.
When we have a discrete random variable, which is constant, it takes a constant value.
We can think of this as a special degenerate case of the normal with mean equal to b and with variance equal to 0.
Even though it is discrete, not continuous, we will still think of it as a degenerate type of a normal random variable, and by adopting this convention, then it will always be true that a linear function of a normal random variable is normal, even if a is equal to 0.
Now that we have the definition and some properties of normal random variables, the next question is whether we can calculate probabilities associated with normal random variables.
This will be the subject of the next segment.