Let us now consider a simple example.
Let random variables X and Y be described by a joint PMF which is the one shown in this table.
Question– are X and Y independent? We can try to answer this question by using the definition of independence.
But it is actually more instructive to proceed in a somewhat more intuitive way.
We look at this table, and we observe that the value of one is possible for X.
In particular, the probability that X takes the value of one, this is the marginal probability, this can be found by adding the probabilities of all of the outcomes in this column, which, in this case, is 3 over 20.
Suppose now that somebody tells you the value of Y.
For example, I tell you that Y happens to be equal to one, in which case you are transported into this universe.
In this universe, the conditional probability that X takes a value of one, given that Y takes a value of one, what is it? In this universe, there’s zero probability associated to this outcome.
So this probability is zero, which is different than 3 over 20.
And since these two numbers are different, this means that information from Y changes our beliefs about what’s going to happen to X.
And so, we do not have independence.
So again, intuitively, in the beginning, we thought that X equal to one was possible.
But information given by Y, namely that Y is equal to one, tells us that, actually, X equals to one is impossible.
Information about Y changed our beliefs about X, so X and Y are dependent.
Now, when we first introduced the notion of independence some time ago, we also introduced the notion of conditional independence.
And we said that conditional independence is the same as ordinary independence, except that it would be applied to a conditional universe.
Something similar can be done for the case of random variables as well.
So suppose, for example, that someone tells us that the outcome of the experiment was such that it belongs to this blue set.
This is the set where X is less than or equal to 2, and Y is larger than or equal to three.
So we’re given this information, and this is now our new conditional model.
The question is, within this new conditional model are random variables X and Y independent? Let’s just right down the conditional model, where I’m only showing the four possible outcomes that are allowed in the conditional model.
All the others, of course, will have zero probability in the conditional model.
So in the conditional model, probabilities will keep the same proportions as in the unconditional model– and the proportions are 1, 2, 2, 4– but then they need to be scaled, or normalized, so that they add to 1.
And to make them add to 1, we need to divide them all by 9.
In this conditional model, this is the joint PMF of the two random variables X and Y.
Let us find the marginal PMFs.
To find the marginal PMF of X, we add the entries in this column.
And we get here 1/3, and here 2/3.
And to find the marginal PMF of y, we add the entries in this [row] to find 2/3.
And we adds the entries in that [row] to find 1/3.
So this is the marginal PMF of x.
That’s the marginal PMF of Y.
And now we notice that this entry of the joint PMF is 1/3 times 1/3, the product of the marginals.
This entry is the product of 1/3 times 2/3, the product of the marginals, and so on for the remaining entries.
So each entry of the joint PMF is equal to the product of the corresponding entries of the marginal PMFs.
And this is the definition of independence.
So in this conditional blue universe, we do have independence.
And the way that this was established was to check that the joint PMF factors as a product of marginal PMFs.