Least mean squares estimation is remarkable because it has such a simple answer. The way to come up with estimates, if what you care about is to keep the mean squared error small, the way to come up with estimates is to just report the conditional expectation, which is going to be a number, once …
After our warm-up, we can now come to the real problem. We have, again, a random variable Theta with a known prior distribution. And we’re interested in a point estimate. What will be different this time, however, is that we now have an observation. And we also have a model of that observation as a …
Probability – L16.3 LMS Estimation of One Random Variable Based on Another Read More »
In this segment, we introduce the subject of least mean squares estimation. But as a warm-up, we will start with a very simple special case. Suppose that we have some random variable that we wish to estimate and that we have the probability distribution of this random variable– a probability mass function if it’s discrete …
Probability – L16.2 LMS Estimation in the Absence of Observations Read More »
We have seen so far two ways of estimating an unknown parameter. We can use the maximum a posteriori probability estimate, or we can use the conditional expectation. That is, the mean of the posterior distribution. These were, in some sense, arbitrary choices. How about imposing a performance criterion, and then finding an estimate which …
Let us now come back to the trajectory estimation problem that we introduced earlier. We have an object that moves vertically. At any given time t, the height at which the object is found is equal to this expression. It corresponds to the following– the object starts at time 0, at some initial height Theta0, …
Probability – L15.8 Trajectory Estimation Illustration Read More »
The trajectory estimation problem that we considered gives you a first glimpse into a large field that deals with linear normal models. In this segment, I will just give you a preview of what happens in that field, although, we will not attempt to prove or justify anything. What happens in this field is that …
In this segment, we introduce a model with multiple parameters and multiple observations. It is a model that appears in countless real-world applications. But instead of giving you a general and abstract model, we will talk about a specific context that has all the elements of the general model, but it has the advantage of …
Probability – L15.6 Multiple Parameters; Trajectory Estimation Read More »
We now continue our discussion of the model in which we obtain several measurements of an unknown random variable Theta in the presence of additive noise, under the same assumptions as before. Theta and Wi are all independent random variables. And they’re also normal. We have seen that in this case, the posterior distribution of …
We are now ready to move on to a model which is quite interesting and quite realistic. This is a model in which we have an unknown parameter modeled as a random variable that we try to estimate. This is the random variable, Theta. And we have multiple observations of that random variable. Each one …
Probability – L15.4 The Case of Multiple Observations Read More »
Introduction As a preparation for more complex and more difficult models, we will start by looking at the simplest model that there is, that involves a linear relation and normal random variables. Model The specifics of the model are as follows. There’s an unknown parameter modeled as a random variable, Theta, that we wish to …