The mathematics of the correlation coefficient are important.
But it is perhaps more important to be able to interpret it correctly.
A correlation coefficient of let’s say 0.
5, tells us that something interesting is going on as far as the relation of X and Y is concerned.
But what exactly?
It tells us that the two random variables are associated in some sense.
But this is often misinterpreted to mean that there is a causal relation between the two.
But this is wrong.
A large correlation coefficient in general does not indicate that there is a causal relation between the random variables.
As an example, suppose that X somehow quantifies the mathematical aptitude of a person.
And Y somehow quantifies the musical ability of a person.
In general, it has been found that mathematical aptitude and musical ability are correlated.
People who score high on one will score high on the other as well.
Is there a causal relation?
If you study math a lot and you become very good at math, does it mean that you would become a better musician?
Not necessarily.
Or if you practice the violin day in and day out, does it mean that you will score better in the math exam?
Again, not necessarily.
Perhaps what is going on is that there’s a certain feature of the human brain and when that feature is well developed, then that feature helps both in math and in musical ability.
And this is a typical situation of how a correlation coefficient may arise.
That is often a correlation coefficient that’s significant, reflects that there is an underlying common but perhaps hidden factor that affects both of the random variables X and Y.
Let’s us go through a simple numerical example that models a situation of this kind.
Suppose that Z, V, and w are independent random variables.
And that we have two more random variables defined by these relations.
Not that there’s no direct influence from X to Y or from Y to X.
But on the other hand, there’s a common underlying factor, this random variable Z that affects both X and Y.
Because of this, we expect that X and Y will somehow have some kind of relation or association between them.
And we would like to measure the strength of that association.
The way to measure it will be in terms of the correlation coefficient, which we will now proceed to compute.
To have a complete example in our hands and in order to also keep things simple, let’s us assume that the basic underlying random variable Z, V, and W all have 0 means and unit variances.
And now let us take the definition of the correlation coefficient and start calculating.
Let us look at the variance of X.
Because X is the sum of two independent random variables, its variance is going to be the sum of those variances.
And we have assumed that each one of those variances is equal to 1.
So the variance of X is equal to 2.
And that implies that the standard deviation of X is equal to the square root of 2.
By a similar argument, the standard deviation of Y is also equal to the square root of 2.
Now, let us look at the covariance between X and Y.
Because X and Y have 0 means, the covariance is just the expected value of the product of the two random variables.
And using the definition of what these two random variables are, it’s this particular product here.
We expand the product into a sum of four terms.
And take the expected value of each one of the four terms.
Which leaves us with this particular expression here.
Now, Z has 0 mean and unit variance.
Therefore, the expected value of Z squared is equal to 1.
How about the next term?
V and Z are independent.
So the expected value of the product is the product of the expected values.
But the expected values are zero, so this term is zero.
And with a similar argument, the other terms are zero as well.
So the co-variance is equal to 1.
And from this, we can conclude our calculation and write that the correlation coefficient between X and Y is equal to 1 divided by the square root of 2 times square root of 2, which is 1/2.
This example also serves to give you a rough idea of what it may mean to have a correlation coefficient of 1/2.
It means that the two random variables have some common elements.
And they also have some idiosyncratic elements.
And these two elements are roughly equal in weight.
If V and W were completely absent, the correlation coefficient would have been 1.
If on the other hand V and W had a huge variance, so as to completely hide the effect of Z, then the value of the correlation coefficient would have been much, much smaller perhaps closer to 0.
And in the extreme case of course where Z is completely absent, then X and Y are independent, and we get a correlation coefficient of 0.