The covariance between two random variables tells us something about the strength of the dependence between them.
But it is not so easy to interpret qualitatively.
For example, if I tell you that the covariance of X and Y is equal to 5, this does not tell you very much about whether X and Y are closely related or not.
Another difficulty is that if X and Y are in units, let’s say, of meters, then the covariance will have units of meters squared.
And this is hard to interpret.
A much more informative quantity is the so-called correlation coefficient, which is a dimensionless version of the covariance.
It is defined by this formula here.
We just take the covariance and divide it by the product of the standard deviations of the two random variables.
Now, if X has units of meters, then the standard deviation also has units of meters.
And so this ratio will be dimensionless.
And it is not affected by the units that we’re using.
The same is true for this ratio here, and this is why the correlation coefficient does not have any units of its own.
One remark– if we’re dealing with a random variable whose standard deviation is equal to 0– so its variance is also equal to 0– then we have a random variable, which is identically equal to a constant.
Well, for such cases of degenerate random variables, then the correlation coefficient is not defined, because it would have involved a division by 0.
A very important property of the correlation coefficient is the following.
It turns out that the correlation coefficient is always between minus 1 and 1.
And this allows us to judge whether a certain correlation coefficient is big or not, because we now have an absolute scale.
And so it does provide a measure of the degree to which two random variables are associated.
To interpret the correlation coefficient, let’s now look at some extreme cases.
Suppose that X and Y are independent.
In that case, we know that the covariance is going to be equal to 0.
And therefore, the correlation coefficient is also going to be equal to 0.
And in that case, we say that the two random variables are uncorrelated.
However, the converse statement is not true.
We have seen already an example in which we have zero covariance and therefore zero correlation, but yet the two random variables were dependent.
Let us now look at the other extreme, where the two random variables are as dependent as they can be.
So let’s look at the correlation coefficient of one random variable with itself.
What is it going to be?
The covariance of a random variable with itself is just the variance of that random variable, now, sigma X is going to be the same as sigma Y, because we’re taking Y to be the same as X.
So we’re dividing by sigma X squared.
But the square of the standard deviation is the variance.
So we obtain a value of 1.
So a correlation coefficient of 1 shows up in such a case of an extreme dependence.
If instead we had taken the correlation coefficient of X with the negative of X, in that case, we would have obtained a correlation coefficient of minus 1.
A somewhat more general situation than the one we considered here is the following.
If we have two random variables that have a linear relationship– that is, if I know Y I can figure out the value of X with absolute certainty, and I can figure it out by using a linear formula.
In this case, it turns out that the correlation coefficient is either plus 1 or minus 1.
And the converse is true.
If the correlation coefficient has absolute value of 1, then the two random variables obey a deterministic linear relation between them.
So to conclude, an extreme value for the correlation coefficient of plus or minus 1 is equivalent to having a deterministic relation between the two random variables involved.
A final remark about the algebraic properties of the correlation coefficient- What can we say about the correlation coefficient of a linear function of a random variable with another?
Well, we already know something about what happens to the covariance when we form a linear function.
And the covariance of aX plus b with Y is related this way to the covariance of X with Y.
Now, let us use this property and calculate the correlation coefficient between aX plus b and Y.
In the numerator, we have the covariance of aX plus b with Y, which is equal to a times the covariance of X with Y.
At the denominator, we have the standard deviation of this random variable.
Now, the standard deviation of this random variable is equal to a times the standard deviation of X, if a is positive.
If a is negative, then we need to put the minus sign.
But in either case, we will have here the absolute value of a times the standard deviation of X.
And then we divide by the standard deviation of the second random variable, which is Y.
And so what we obtain here is this ratio, which is a correlation coefficient of X with Y times this quantity, which is the sign of a.
So we have the sign of a times the correlation coefficient of X with Y.
So in particular, the magnitude of the correlation coefficient is not going to change when we replace X by aX plus b.
And this essentially means that if we change the units of the random variable X, for example, suppose that X was degrees Celsius and aX plus b is degrees Fahrenheit, going from one set of units, Celsius degrees, to another set of units, degrees in Fahrenheit, is not going to change the correlation coefficient of the temperature with some other random variable.
So this is a nice property of the correlation coefficient, again, which reflects the fact that it’s dimensionless, it doesn’t have any units of its own, and it doesn’t depend on what kinds of units we use for each one of the random variables.