We now come to a very important concept, the concept of independence of random variables.
We are already familiar with the notion of independence of two events.
We have the mathematical definition, and the interpretation is that conditional probabilities are the same as unconditional ones.
Intuitively, when you are told that B occurred, this does not change your beliefs about A, and so the conditional probability of A is the same as the unconditional probability.
We have a similar definition of independence of a random variable and an event A.
The mathematical definition is that event A and the event that X takes on a specific value, that these two events are independent in the ordinary sense.
So the probability of both of these events happening is the product of their individual probabilities.
But we require this to be true for all values of little x.
Intuitively, if I tell you that A occurred, this is not going to change the distribution of the random variable x.
This is one interpretation of what independence means in this context.
And this has to be true for all values of little x, that is, when [the] event occurs, the probabilities of any particular little x [are] going to be the same as the original unconditional probabilities.
We also have a symmetrical interpretation.
If I tell you the value of X, then the conditional probability of event A is not going to change.
It’s going to be the same as the unconditional probability.
And again, this is going to be the case for all values of X.
So, no matter what they tell you about X, your beliefs about A are not going to change.
We can now move and define the notion of independence of two random variables.
The mathematical definition is that the event that X takes on a value little x and the event that Y takes on a value little y, these two events are independent, and this is true for all possible values of little x and little y.
In PMF notation, this relation here can be written in this form.
And basically, the joint PMF factors out as a product of the marginal PMFs of the two random variables.
Again, this relation has to be true for all possible little x and little y.
What does independence mean? When I tell you the value of y, and no matter what value I tell you, your beliefs about X will not change.
So that the conditional PMF of X given Y is going to be the same as the unconditional PMF of X.
And this has to be true for any values of the arguments of these PMFs.
There is also a symmetric interpretation, which is that the conditional PMF of Y given X is going to be the same as the unconditional PMF of Y.
We have the symmetric interpretation because, as we can see from this definition, X and Y have symmetric roles.
Finally, we can define the notion of independence of multiple random variables by a similar relation.
Here, the definition is for the case of three random variables, but you can imagine how the definition for any finite number of random variables will go.
Namely, the joint PMF of all the random variables can be expressed as the product of the corresponding marginal PMFs.
What is the intuitive interpretation of independence here? It means that information about some of the random variables will not change your beliefs, the probabilities, about the remaining random variables.
Any conditional probabilities and any conditional PMFs will be the same as the unconditional ones.
In the real world, independence models situations where each of the random variables is generated in a decoupled manner, in a separate probabilistic experiment.
And these probabilistic experiments do not interact with each other and have no common sources of uncertainty.