Independence is one of the central concepts of probability theory, because it allows us to build large models from simpler ones.
How should we define independence in the continuous case? Our guide comes from the discrete definition.
By analogy with the discrete case, we will say that two jointly continuous random variables are independent if the joint PDF is equal to the product of the marginal PDFs.
We can now compare with the multiplication rule, which is always true as long as the density of Y is positive.
So this is always true.
In the case of independence, this is true.
So in the case of independence, we must have that this term is equal to that term, at least whenever this quantity– the marginal of Y– is positive.
So to restate it, independence is equivalent to having the conditional, given Y, be the same as the unconditional PDF of X.
And this has to be true whenever Y has a positive density so that this quantity is well defined, and it also has to be true for all xs.
Now, what does this really mean? The conditional PDF, as we have discussed, in terms of pictures, is a slice of the joint PDF.
Therefore, independence is the same as requiring that all of the slices of the joint have the same shape, and it is the shape of the marginal PDF.
For a more intuitive interpretation, no matter what value of Y you observe, the distribution of X does not change.
In this sense, Y does not convey any information about X.
Notice also that this definition is symmetric as far as X and Y are concerned.
So by symmetry, when we have independence, it also means that X does not convey any information about Y, and that the conditional density of Y, given X, has to be the same as the unconditional density of Y.
We can also define independence of multiple random variables.
The definition is the obvious one.
The joint PDF of all the random variables involved must be equal to the product of the marginal PDFs.
Intuitively, what that means is that knowing the values of some of the random variables does not affect our beliefs about the remaining random variables.
Finally, let us note some consequences of independence, which are identical to the corresponding properties that we had in the discrete case, and the proofs are also exactly the same.
So the expectation of the product of independent random variables is the product of the expectations, the variance of the sum of independent random variables is the sum of the variances, and functions of independent random variables are also independent, which, in particular, implies, using the previous rule, that the expected value of a product of this kind is going to be the product of these expectations.
So independence of continuous random variables is pretty much the same as independence of discrete random variables as far as mathematics are concerned, and the intuitive content of the independence assumption is the same as in the discrete case.
One random variable does not provide any information about the other.