Let us now revisit the variance and see what happens in the case of independence.
Variances have some general properties that we have already seen.
However, since we often add random variables, we would like to be able to say something about the variance of the sum of two random variables.
Unfortunately, the situation is not so simple, and in general, the variance of the sum is not the same as the sum of the variances.
We will see an example shortly.
On the other hand, when X and Y are independent, the variance of the sum is equal to the sum of the variances, and this is a very useful fact.
Let us go through the derivation of this property.
But to keep things simple, let us assume just for the sake of the derivation, that the two random variables have 0 mean.
So in that case, the variance over the sum is just the expected value of the square of the sum.
And we can expand the quadratic and write this as the expectation of X squared plus 2 X Y plus Y squared.
Then we use linearity of expectations to write this as the expected value of X squared plus twice the expected value of X times Y and then plus the expected value of Y squared.
Now, the first term is just the variance of X because we have assumed that we have 0 mean.
The last term is similarly the variance of Y.
How about the middle term? Because of independence, the expected value of the product is the same as the product of the expected values, and the expected values are 0 in our case.
So this term, because of independence, is going to be equal to 0.
In particular, what we have is that the expected value of XY equals the expected value of X times the expected value of Y, equal to 0.
And so we have verified that indeed the variance of the sum is equal to the sum of the variances.
Let us now look at some examples.
Suppose that X is the same random variable as Y.
Clearly, this is a case where independence fails to hold.
If I tell you the value of X, then you know the value of Y.
So in this case, the variance of the sum is the same as the variance of twice X.
Since X is the same as Y, X plus Y is 2 times X.
And then using this property for the variance, what happens when we multiply by a constant? This is going to be 4 times the variance of X.
In another example, suppose that X is the negative of Y.
In that case, X plus Y is identically equal to 0.
So we’re dealing with a random variable that takes a constant value.
In particular, it is always equal to its mean, and so the difference from the mean is always equal to 0, and so the variance will also evaluate to 0.
So we see that the variance of the sum can take quite different values depending on the sort of interrelation that we have between the two random variables.
So these two examples indicate that knowing the variance of each one of the random variables is not enough to say much about the variance of the sum.
The answer will generally depend on how the two random variables are related to each other and what kind of dependencies they have.
As a last example, suppose now that X and Y are independent.
X is independent from Y, and therefore X is also independent from minus 3Y.
Therefore, this variance is equal to the sum of the variances of X and of minus 3Y.
And using the facts that we already know, this is going to be equal to the variance of X plus 9 times the variance of Y.
As an illustration of the usefulness of the property of the variance that we have just established, we will now use it to calculate the variance of a binomial random variable.
Remember that a binomial with parameters n and p corresponds to the number of successes in n independent trials.
We use indicator variables.
This is the same trick that we used to calculate the expected value of the binomial.
So the random variable X sub i is equal to 1 if the i-th trial is a success and is a 0 otherwise.
And as we did before, we note that X, the total number of successes, is the sum of those indicator variables.
Each success makes one of those variables equal to 1, so by adding those indicator variables, we’re just counting the number of successes.
The key point to note is that the assumption of independence that we’re making is essentially the assumption that these random variables Xi are independent of each other.
So we’re dealing with a situation where we have a sum of independent random variables, and according to what we have shown, the variance of X is going to be the sum of the variances of the Xi’s.
Now, the Xi’s all have the same distribution so all these variances will be the same.
It suffices to consider one of them.
Now, X1 is a Bernoulli random variable with parameter p.
We know what its variance is– it is p times 1 minus p.
And therefore, this is the formula for the variance of a binomial random variable.