We have defined the conditional expectation of a random variable given another as an abstract object, which is itself a random variable.
Let us now do something analogous with the notion of [the] conditional variance.
Let us start with the definition of the variance, which is the following.
We look at the deviation of the random variable from its mean, square it, then take the average of that quantity.
If we live in a conditional universe where we are told the value of some other random variable, capital Y, then inside that conditional universe the variance becomes the following.
It is defined the same way.
Well, in the conditional universe, this is the expected value of X.
So this quantity here is the deviation of X from its expected value in that conditional universe.
We square this quantity, we find the squared deviation, and we look at the expected value of that squared deviation.
But because we live in a conditional universe, of course, this expectation has to be a conditional one given the information that we have available.
So this is nothing but the ordinary variance, but it’s the variance of the conditional distribution of the random variable, capital X.
This is an equality between numbers.
If I tell you the value of little y, the conditional variance is defined by this particular quantity, which is a number.
Now, we proceed in the same way as we proceeded for the case where we defined the conditional expectation as a random variable.
Namely, we think of this quantity as a function of little y, and that function can be now used to define a random variable.
And that random variable, which would denote this way, this is the random variable which takes this specific value when capital Y happens to be equal to little y.
Once we know the value of capital Y, then this quantity takes a specific value.
But before we know the value of capital Y, then this quantity is not known.
It’s random.
It’s a random variable.
Let us look at an example to make this more concrete.
Suppose that Y is a random variable.
We draw that random variable.
And we’re told that conditioned on the value of that random variable, X is going to be uniform on this particular interval from 0 to Y.
So if I tell you that capital Y takes on a specific numerical value, then the random variable X is uniform on the interval from 0 to little y.
A random variable that’s uniform on an interval of length little y has a variance that we know what it is.
It’s y squared over 12.
So this is an equality between numbers.
For any specific value of little y, this is the numerical value of the conditional variance.
Let us now change this equality between numbers into an abstract equality between random variables.
The random variable, variance of X given Y, is a random variable that takes this value whenever capital Y is little y.
But that’s the same as this random variable.
This is a random variable that takes this value whenever capital Y happens to be equal to little y.
So we have defined the abstract concept of a conditional variance, similar to the case of conditional expectations.
For conditional expectations, we had the law of iterated expectations.
That tells us that the expected value of the conditional expectation is the unconditional expectation.
Is it true that the expected value of the conditional variance is going to be the same as the unconditional variance?
Unfortunately, no.
Things are a little more complicated.
The unconditional variance is equal to the expected value of the conditional variance, but there is an extra term, that is, the variance of the conditional expectation.
The entries here in red are all random variables.
So the conditional variance has been defined as a random variable, so it has an expectation of its own.
The conditional expectation, as we have already discussed, is also a random variable, so it has a variance of its own.
And by adding those terms, we get the total variance of the random variable X.
So what we will do next will be first to prove this equality, and then give a number of examples that are going to give us some intuition about what these terms mean and why this equality makes sense.