Just in order to get some more familiarity with joint PDFs, let us look at independent normals.
Actually, this is an important example because noise is often modeled by normal random variables, and noise terms that show up at different parts of a system, or at different times, are often assumed to be independent.
Suppose that we have two standard normal random variables, X and Y, with zero means and unit variances.
If their independent, their joint PDF is the product of the marginal PDFs and takes this form.
This is just the PDF of a standard normal X and the PDF of a standard normal Y and we multiply them.
If we are to plot this joint PDF we obtain this figure.
It looks like a bell which is centered at the origin– at the point with coordinates zero, zero.
One way to think about what is going on here is to rewrite this expression as 1 over 2pi, and then the exponential of minus 1/2 x squared plus y squared.
If we look at the unit circle in xy space, which is the set of points at which x squared plus y squared is equal to 1, then, on that circle, the PDF takes a constant value because this quantity is constant on that circle.
And the same is true for any other circle.
On any circle the PDF takes a constant value, of course, a different constant.
So the circles centered at the origin are the so-called contours of the joint PDF.
On each contour the joint PDF is a constant.
Let us now generalize.
Consider two independent normal random variables, but with general means mu x and mu y, and variances sigma x squared and sigma y squared.
The joint is, again, the product of the marginal PDFs and, therefore, takes this form.
This looks intimidating but, in fact, it is pretty simple.
This part is just a normalizing constant.
It is the constant that’s needed so that the joint PDF integrates to 1.
What we have here is the negative exponential of a quadratic function of x and y.
Let us plot the contours of this quadratic.
Remember that contour is the set of points where the quadratic takes a constant value.
And by consequence, the joint PDF also takes a constant value.
If you have set this quadratic to a constant, what you have is the equation that describes an ellipse.
And it is an ellipse whose principal axes run along the x and y directions, and those ellipses are all centered at this particular point, mu x, mu y.
The joint PDF is largest when the exponent is equal to zero.
And this happens when x is equal to mu x, and y is equal to mu y.
That is, right at the center of the ellipse.
That’s where the joint PDF is largest.
As you move to ellipses that are further out on this outer ellipse, this expression is a constant.
It’s the exponential of the negative of some positive numbers.
So you get a smaller value for the joint PDF.
If you move to a further ellipse further out, then again, the joint PDF will be a constant, but it’s going to be a smaller constant.
Now, for the case of standard normals, the joint PDF was circularly symmetric.
The contours were actually circles, instead of ellipses.
But this is not the case in general.
For example, suppose that the variance of Y is bigger than the variance of X.
Then you get a shape as the one shown in this figure.
Since the variance of Y is larger, we expect Y to take values over a bigger range, and to be larger typically than the values of X.
And so the bell shape that we have for the joint PDF is stretched in the y direction.
It extends further out in the y direction than it does in the x direction.
To conclude, the joint PDF of two independent normals has the shape of a bell.
The center of the bell is determined by the means.
Furthermore, the bell is stretched in the x and y directions by an amount that is determined by the variances of x and y.
However, the stretching is always along the coordinate axes.
If you wanted a bell that stretches in some diagonal direction, or if you have contours that are ellipses but with some different kinds of axes, then you will have dependence between the two random variables.
In that case, we will be dealing with a so-called bivariate normal distribution, but we will not pursue this any further at this point.