We will now talk about conditional expectations of one random variable given another.
As we will see, there will be nothing new here, except for older results but given in new notation.
Any PMF has an associated expectation.
And so conditional PMFs also have associated expectations, which we call conditional expectations.
We have already seen them for the case where we condition on an event, A.
The case where we condition on random variables is exactly the same.
We let the event, A, be the event that Y takes on a specific value.
And then we calculate the expectation using the relevant conditional probabilities, those that are given by the conditional PMF.
So the conditional expectation of X given that Y takes on a certain value is defined as the usual expectation, except that we use the conditional probabilities that apply given that Y takes on a specific value little y.
Recall now the expected value rule for ordinary expectations.
And also the Expected Value Rule for conditional expectations given an event, something that we have already seen.
Now, in PMF notation, the expected value rule takes a similar form.
The event, A is replaced by the specific event that Y takes on a specific value.
And in that case, the conditional PMF given the event A is just the conditional PMF given that random variable Y takes on a specific value, little y.
For the case where we condition on events, we also developed a version of the total probability theorem and the total expectation theorem.
We can do the same when we condition on random variables.
So suppose that the sample space has been partitioned into n, disjoint scenarios.
The total probability theorem tells us that the probability of the event that random variable X takes on a value little x, can be found by taking the probabilities of this event under each one of the possible scenarios.
And then weighing those probabilities according to the probabilities of the different scenarios.
Now, suppose that we are dealing with a random variable that takes values in a set consisting of n elements.
And let us consider scenarios Ai, the i-th scenario is the event that the random variable Y takes on the i-th possible value.
We can apply the total probability theorem to this situation.
We can find the probability that the random variable X takes on a certain value, little x, by considering the probability of this event happening under each possible scenario, where a scenario is that Y took on a specific value, and then weigh those probabilities according to the probabilities of the different scenarios.
The story with the total expectation theorem is similar.
We know that an expectation can be found by taking the conditional expectations under each one of the scenarios and weighing them according to the probabilities of the different scenarios.
Again, let the event that Y takes on a specific value be a different scenario.
And with this correspondence we obtain the following version of the total expectation theorem.
We have a sum of different terms.
And each term in the sum is the probability of a given scenario times the expected value of X under this particular scenario.
At this point, I have to add a comment of a more mathematical flavor.
We have been talking about a partition of the sample space into finitely many scenarios.
But if Y takes on values in a discrete but infinite set, for example, if Y can take on any integer value, the argument that we have given is not quite complete.
Fortunately, the total probability theorem and the total expectation theorem, they both remain true, even for the case where Y ranges over an infinite set as long as the random variable X has a well-defined expectation.
For the total probability theorem, the proof for the general case can be carried out without a lot of difficulty, just using the countable additivity axiom.
However, for the total expectation theorem, it takes some harder mathematical work.
And this is beyond our scope.
But we will just take this fact for granted, that the total expectation theorem carries over to the case where we’re adding over an infinite sequence of possible values of Y.
In the rest of the course we will often use the total expectation theorem, including in cases where Y ranges over an infinite discrete set.
In fact, we will see that this theorem is an extremely useful tool that can be used to divide and conquer complicated models.