The trajectory estimation problem that we considered gives you a first glimpse into a large field that deals with linear normal models.
In this segment, I will just give you a preview of what happens in that field, although, we will not attempt to prove or justify anything.
What happens in this field is that we’re dealing with models where there are some underlying independent normal random variables.
And then, the random variables of interest, the unknown parameters and the observations, can all be expressed as linear functions of these independent normals.
Since linear functions of independent normals are normal, in particular, this means that the Theta j and the Xi are all normal random variables.
Carrying out inference within this class of models goes under the name of linear regression.
One can proceed using pretty much the same steps as we had in the trajectory estimation problem and write down formulas for the posterior.
And it turns out that in every case, the posterior of the parameter vector takes a form which is the exponential of and the negative of a quadratic function of the parameters.
And this means, in particular, that in order to find the MAP estimate of the vector Theta, what we need to do is to just minimize this quadratic function with respect to theta.
And minimizing a quadratic function is done by taking derivatives and setting them to zero which leads us to a system of linear equations, exactly as in the trajectory inference problem.
And this means that numerically it is very simple to come up with a MAP estimate.
There’s an interesting fact that comes out of the algebra involved, namely, that the MAP estimate of each one of the parameters turns out to also be a linear function of the observations.
We saw this property in the estimators that we derived for simpler cases in this lecture sequence.
It turns out that this property is still true.
And this is an appealing and desirable property because it means that these estimators can be applied very efficiently in practice without having to do any complicated calculations.
Finally, there’s a number of important facts, some of which we have seen in our examples which are true in very big generality.
One fact is that the maximum a posteriori probability estimate of some parameter turns out to be the same as the conditional expectation of that parameter.
Furthermore, if you look at this joint density of all the Theta parameters and from it you find the marginal density of the Theta parameters always within this conditional universe.
It turns out that this marginal posterior PDF is itself normal.
Since it is normal, its mean– which is this quantity– is going to be equal to its peak.
And therefore, it is equal also to the MAP estimate that would be derived from this marginal PDF.
So what do we have here?
There are two ways of coming up with MAP estimates.
One way is to find the peak of the joint PDF, and then read out the different components of Theta.
Another way of coming up with MAP estimates is to find for each parameter the marginal PDF and look at the peak of this marginal PDF.
It turns out that for this model these two approaches are going to give you the same answer.
Whether you work with the marginal or with the joint, you get the same MAP estimates.
And this is a reassuring property to have.
Finally, as in the examples that we have worked in more detail, it turns out that the mean squared error conditioned on a particular observation is the same no matter what the value of that observation was.
And furthermore, there are fairly simple and easily computable formulas that one can apply in order to find what this mean squared error is.
So to summarize, this class of models that involve linear relations and normal random variables have a rich set of important and elegant properties.
This is one of the reasons why these models are used very much in practice.
And they’re probably the most widely used class of statistical models.