In this segment, we pursue two themes.
Every concept has a conditional counterpart.
We know about PDFs, but if we live in a conditional universe, then we deal with conditional probabilities.
And we need to use conditional PDFs.
The second theme is that discrete formulas have continuous counterparts in which summations get replaced by integrals, and PMFs by PDFs.
So let us recall the definition of a conditional PMF, which is just the same as an ordinary PMF but applied to a conditional universe.
In the same spirit, we can start with a PDF, which we can interpret, for example, in terms of probabilities of small intervals.
If we move to a conditional model in which event A is known to have occurred, probabilities of small intervals will then be determined by a conditional PDF, which we denote in this manner.
Of course, we need to assume throughout that the probability of the conditioning event is positive so that conditional probabilities are well-defined.
Let us now push the analogy further.
We can use a PMF to calculate probabilities.
The probability that X takes [a] value in a certain set is the sum of the probabilities of all the possible values in that set.
And a similar formula is true if we’re dealing with a conditional model.
Now, in the continuous case, we use a PDF to calculate the probability that X takes values in a certain set.
And by analogy, we use a conditional PDF to calculate conditional probabilities.
We can take this relation here to be the definition of a conditional PDF.
So a conditional PDF is a function that allows us to calculate probabilities by integrating this function over the event or set of interest.
Of course, probabilities need to sum to 1.
This is true in the discrete setting.
And by analogy, it should also be true in the continuous setting.
This is just an ordinary PDF, except that it applies to a model in which event A is known to have occurred.
But it still is a legitimate PDF.
It has to be non-negative, of course.
But also, it needs to integrate to 1.
When we condition on an event and without any further assumption, there’s not much we can say about the form of the conditional PDF.
However, if we condition on an event of a special kind, that X takes values in a certain set, then we can actually write down a formula.
So let us start with a random variable X that has a given PDF, as in this diagram.
And suppose that A is a subset of the real line, for example, this subset here.
What is the form of the conditional PDF? We start with the interpretation of PDFs and conditional PDFs in terms of probabilities of small intervals.
The probability that X lies in a small interval is equal to the value of the PDF somewhere in that interval times the length of the interval.
And if we’re dealing with conditional probabilities, then we use the corresponding conditional PDF.
To find the form of the conditional PDF, we will work in terms of the left-hand side in this equation and try to rewrite it.
Let us distinguish two cases.
Suppose that little X lies somewhere out here, and we want to evaluate the conditional PDF at that point.
So trying to evaluate this expression, we consider a small interval from little x to little x plus delta.
And now, let us write the definition of a conditional probability.
A conditional probability, by definition, is equal to the probability that both events occur divided by the probability of the conditioning event.
Now, because the set A and this little interval are disjoint, these two events cannot occur simultaneously.
So the numerator here is going to be 0.
And this will imply that the conditional PDF is also going to be 0.
This, of course, makes sense.
Conditioned on the event that X took values in this set, values of X out here cannot occur.
And therefore, the conditional density out here should also be 0.
So the conditional PDF is 0 outside the set A.
And this takes care of one case.
Now, the second case to consider is when little x lies somewhere inside here inside the set A.
And in that case, our little interval from little x to little x plus delta might have this form.
In this case, the intersection of these two events, that X lies in the big set and X lies in the small set, the intersection of these two events is the event that X lies in the small set.
So the numerator simplifies just to the probability that the random variable X takes values in the interval from little x to little x plus delta.
And then we rewrite the denominator.
Now, the numerator is just an ordinary probability that the random variable takes values inside a small interval.
And by our interpretation of PDFs, this is approximately equal to the PDF evaluated somewhere in that small interval times delta.
At this point, we notice that we have deltas on both sides of this equation.
By cancelling this delta with that delta, we finally end up with a relation that the conditional PDF should be equal to this expression that we have here.
So to summarize, we have shown a formula for the conditional PDF.
The conditional PDF is 0 for those values of X that cannot occur given the information that we are given, namely that X takes values at that interval.
But inside this interval, the conditional PDF has a form which is proportional to the unconditional PDF.
But it is scaled by a certain constant.
So in terms of a picture, we might have something like this.
And so this green diagram is the form of the conditional PDF.
The particular factor that we have here in the denominator is exactly that factor that is required, the scaling factor that is required so that the total area under the green curve, under the conditional PDF is equal to 1.
So we see once more the familiar theme, that conditional probabilities maintain the same relative sizes as the unconditional probabilities.
And the same is true for conditional PMFs or PDFs, keeping the same shape as the unconditional ones, except that they are re-scaled so that the total probability under a conditional PDF is equal to 1.
We can now continue the same story and revisit everything else that we had done for discrete random variables.
For example, we have the expectation of a discrete random variable and the corresponding conditional expectation, which is just the same kind of object, except that we now rely on conditional probabilities.
Similarly, we can take the definition of the expectation for the continuous case and define a conditional expectation in the same manner, except that we now rely on the conditional PDF.
So this formula here is the definition of the conditional expectation of a continuous random variable given a particular event.
We have a similar situation with the expected value rule, which we have already seen for discrete random variables in both of the unconditional and in the conditional setting.
We have a similar formula for the continuous case.
And at this point, you can guess the form that the formula will take in the continuous conditional setting.
This is the expected value rule in the conditional setting, and it is proved exactly the same way as for the unconditional continuous setting, except that here in the proof, we need to work with conditional probabilities and conditional PDFs, instead of the unconditional ones.
So to summarize, there is nothing really different when we condition on an event in the continuous case compared to the discrete case.
We just replace summations with integrations.
And we replace PMFs by PDFs.