In this segment, we point out and discuss some important but also intuitive properties of conditional expectations.
The first property is the one that is written up here.
What is the intuitive meaning?
If you condition on Y, then the value of Y is known, and so g of Y is also known.
There’s no randomness in it.
It can be treated as a constant, and therefore it can be pulled outside the expectation.
So that’s the intuition.
How does one establish such a result formally?
Let us take the discrete case.
So let us assume that X and Y are both discrete.
What does it take to establish a fact of this kind?
We want to show that two random variables are equal, and that amounts to the following.
We consider an outcome of the experiment, and we want to show that whatever the outcome of the experiment is, these two random variables will be the same.
So let us consider an outcome for which the random variable, Y, takes a specific value, little y.
And of course, this has to be a specific little y that is possible.
Otherwise, conditioning on that event would not be meaningful.
So if an outcome has this value for the random variable, Y, then what does the random variable do?
This is, by definition, the random variable that takes the value– expected value of g Y X, conditional expectation, given that capital Y took on this value.
This was our definition of the concept of the abstract conditional expectation as a random variable.
This is the random variable that takes this specific numerical value whenever the random variable, capital Y, takes the value, little y.
And similarly, if the random variable, capital Y, takes the value, little y, this random variable here is the expected value of X, given that Y is little y.
And when capital Y takes the value, little y, this function, g of Y, takes on this particular numerical value.
So we want to show that these two expressions we will be equal no matter what capital Y is.
Now, when we place ourselves in a conditional universe, where capital Y takes a value, little y, then the joint PMF of X and Y gets concentrated on those values of capital Y that obey this relation.
So conditioned on this event, capital Y is, with certainty, equal to little y.
Therefore, this random variable here, in the conditional universe, is the same as this number.
But now, since this is a number, it can be pulled outside the expectation.
So we have concluded that for any outcome for which the random variable, capital Y, takes this specific value, little y, this random variable takes this value.
This random variable takes this value.
They are the same.
So no matter what the outcome is, these two random variables take the same value, and therefore they are the same random variables.
Now, this is a correct proof if the random variables are discrete.
If the random variables are continuous or general, then carrying out a rigorous proof is actually quite subtle, and it is beyond our scope.
However, the intuition is still correct, and the result is correct.
And we will be using it freely whenever we need to.
Let us now move to a second observation.
Suppose that h is an invertible function.
What does that mean?
That if I give you the value of h, you can tell me the value of the argument.
So in some sense, Y and h of Y can be recovered from each other.
If I give you Y, you can calculate h of Y.
But also, if I give you h of Y, you can figure out what Y was.
An example could be the function, h of Y, equals Y to the third power.
If I tell you the value of Y, you know Y cubed.
But if I tell you Y cubed, you can also figure out Y.
So Y and Y cubed carry exactly the same information.
In that case, the conditional expectation– what you expect, on the average, X to be– if I tell you the value of Y, should be the same as what you would expect X to be if I give you the value of, let’s say, Y cubed.
In both cases, I’m giving you the same amount of information, so the conditional distribution of X should be the same.
And the conditional expectations should also be the same.
So this is, again, a very intuitive fact.
How do we verify that this fact is true?
Using the same method as before.
So fix some particular outcome for which the random variable, capital Y, takes a specific value, little y.
When that happens, this random variable will take this value here.
That’s just by the definition of conditional expectation.
This is the random variable that takes this value whenever capital Y happens to be equal to little y.
In that case, we also have that h of capital Y takes on a specific value, h of little y.
When this random variable takes this specific value, this random variable here will take a value of this kind.
So this is the random variable that takes this value whenever h of capital Y happens to be this specific number.
But now, the event that h of Y takes this specific value, because the function, h, is invertible, is identical to the event that Y takes that particular value.
And so, since this event is identical to that event, the conditional probabilities, given this event, would be the same as the conditional probabilities given that the event.
And therefore, the conditional expectations would also be the same.
Once more, this is a proof that’s entirely rigorous if we are dealing with discrete random variables, although in the continuous case, there could be some subtleties involved.
However, the result is true in general.
The technical details are beyond our scope.