In the next variation we consider, all random variables are continuous.
For this case, we do have a Bayes rule, once more.
And we have worked [out] quite a few examples.
So there’s no point, again, in going through a detailed example.
Let us just discuss some of the issues.
One question is when do these models arise?
One particular class of models that is very useful and very commonly used are so-called linear normal models.
In these models, we, basically, combine various random variables in a linear function.
And all the random variables of interest are now to be normal.
For instance, we might have a signal, a noisy signal, call it Theta, which is now a continuous valued signal.
We receive that signal, but corrupted by some noise, which is independent from what was sent.
And we wish to recover, on the basis of the observation X, we wish to recover the value of Theta.
And then there are versions of this problem that involve Theta vectors instead of single values.
So that Theta consists of multiple components, and where we obtain many measurements X.
We will, actually, see in the next lecture sequence, a quite detailed discussion of models of this type.
And this will be one of our main examples within our study of inference.
There will be another example that we will see a few times, and this involves estimating the parameter of a uniform distribution.
So X is a random variable that’s uniform over a certain range.
But the range itself is random and unknown.
And on the basis of observations X, we would like to make an estimation of what the true value of Theta is.
This is an example that you will see in our collection of solved problems for this class.
So what are the questions in this setting, we wish to come up with ways of estimating Theta, we form an estimator, and the main candidates for estimators at this points are, once more, the maximum a posteriori probability estimator, which looks at this conditional density and picks a value of theta that makes this conditional density as large as possible.
And then the alternative one is the least mean squares estimator, which just computes the expected value of Theta given X.
For any given estimator, we then want to characterize its performance.
In this case, a natural notion of performance is the distance between our estimate, or estimator, from the true value of Theta.
And commonly we use the squared distance and then take the average of that squared distance.
So in a conditional universe where we have already observed some data, we might be interested in this particular expectation, which is the mean squared error of this particular estimator, given that we obtain some particular data.
Or we can average over all possible data points that we might obtain so that we look at the unconditional mean squared error, which is a measure of the overall performance of our estimator.
We will be talking about these criteria and the mean squared error in a fair amount of detail in a subsequent lecture sequence.