In this segment, we discuss a few algebraic properties of the covariance.
There is nothing deep here, only some observations that can be useful if we want to carry out covariance calculations.
We start by looking at this quantity, the covariance of a random variable with itself.
So it’s a special case of this definition but where Y is the same as X.
And therefore, this second term here is the same as the first term.
And so what we’re left with is the expected value of the square deviation of the random variable from its mean.
And we recognize that this is the same as the variance of X.
So we conclude that the covariance of a random variable with itself is the same as the variance.
Now, for the variance, we had an alternative formula, which was often convenient in simplifying variance calculations.
Is there a similar formula for the case of the covariance?
Let us start from the definition and use linearity.
We have a product here of two terms, and we expand that product to obtain four different terms.
The expected value of the sum of these four terms is going to be the sum of their expectations.
So let us go through the steps involved.
We have the expected value of the product of this term with that term.
Gives us expected value of X times Y.
Then we take the expected value of the product of this term with that term.
And because we have a minus sign, we put it out here.
And we have the expected value of X times the expected value of Y inside the expectation.
The next term is going to be the product of this expected value with Y.
And that gives us minus the expected value of X times Y.
And finally, the last term comes by multiplying this quantity with that quantity.
And this is what we have by applying linearity to the definition of the covariance.
Now, remember that the expected value of a random variable is a number, it’s a constant.
And constants can be pulled outside expectations.
So if we do that, what we obtain is the following.
We pull this constant outside the expectation, and we’re left with the expected value of X times the expected value of Y.
Similarly, for the next term, by pulling a constant outside the expectation, we obtain this expression.
And finally, for the last term, we have the expected value of a constant, and this is the same as the constant itself.
We recognize here that the same term gets repeated three times.
And because here we have a minus sign, we can cancel this term with that term.
And what we’re left with is just the difference of these two terms, and this is an alternative form for the covariance of two random variables.
And this form is often easier to work with to calculate covariances compared with the original definition.
Let us now continue with some additional algebraic properties.
Suppose that we know the covariance of X with Y, and we’re interested in the covariance of this linear function of X with Y.
What is the covariance going to be?
To simplify the calculations, let us just assume zero means.
Although, the final conclusion will be the same as in the case of non-zero means.
So in the case of zero means, the covariance of two random variables is just the same as the expected value of the product of the two random variables.
And using linearity, this is the expected value times a of X times Y plus b times the expected value of Y.
Now, we assumed zero means, so this term goes away.
And what we’re left with is a times the covariance of X with Y.
So we see that multiplying X by a increases the covariance by a factor of a.
But adding a constant has no effect.
The reason that it has no effect is that if we take a random variable and add the constant to it, the same constant gets added to its mean.
And so this difference is not affected.
As our final calculation, let us look at the covariance of a random variable with the sum of two other random variables.
Again, we assume zero means.
And so the calculation is as follows.
The covariance is the product of the two random variables involved.
And then we use linearity of expectations to write this as the expected value of X times Y plus the expected value of X times Z.
And we recognize that this is the same as the covariance of X with Y plus the covariance of X with Z.
So in this respect, covariances behave linearly.
They behave linearly with respect to one of the arguments involved.