If our objective is to keep the mean squared estimation error small, then the best possible estimator is the conditional expectation. But sometimes the conditional expectation is hard to calculate. Maybe we’re missing the details of the various probability distributions. Or maybe we have the distributions that we need but the formulas are complicated. After …
In this segment, we’re going to go over a few theoretical properties of the estimation error in least mean squares estimation. Recall that our least mean squares estimator is the conditional expectation of the unknown random variable, given our observations. Let us define the error, which is the difference between the estimator and the random …
Probability – L16.8 Properties of the LMS Estimation Error Read More »
Our discussion of least mean squares estimation so far was based on the case where we have a single unknown random variable and a single observation. And we’re interested in a point estimate of this single unknown random variable. What happens if we have multiple observations or parameters? For example, suppose that instead of a …
Probability – L16.7 LMS Estimation with Multiple Observations or Unknowns Read More »
We now continue with our example and turn to the performance evaluation question. As you recall, we have a Theta that has a certain prior distribution. We’re given a model for the observations. We came up with the joint distribution for X and Theta, which was uniform on this particular shape, and we found that …
Probability – L16.6 Example Continued: LMS Performance Evaluation Read More »
Let us now go through an example. Suppose that we have an unknown random variable Theta that has a uniform distribution between 4 and 10. We observe some other random variable X that’s related to Theta according to the following model. This is the conditional distribution of X given Theta. For any given value of …
Least mean squares estimation is remarkable because it has such a simple answer. The way to come up with estimates, if what you care about is to keep the mean squared error small, the way to come up with estimates is to just report the conditional expectation, which is going to be a number, once …
After our warm-up, we can now come to the real problem. We have, again, a random variable Theta with a known prior distribution. And we’re interested in a point estimate. What will be different this time, however, is that we now have an observation. And we also have a model of that observation as a …
Probability – L16.3 LMS Estimation of One Random Variable Based on Another Read More »
In this segment, we introduce the subject of least mean squares estimation. But as a warm-up, we will start with a very simple special case. Suppose that we have some random variable that we wish to estimate and that we have the probability distribution of this random variable– a probability mass function if it’s discrete …
Probability – L16.2 LMS Estimation in the Absence of Observations Read More »
We have seen so far two ways of estimating an unknown parameter. We can use the maximum a posteriori probability estimate, or we can use the conditional expectation. That is, the mean of the posterior distribution. These were, in some sense, arbitrary choices. How about imposing a performance criterion, and then finding an estimate which …
Let us now come back to the trajectory estimation problem that we introduced earlier. We have an object that moves vertically. At any given time t, the height at which the object is found is equal to this expression. It corresponds to the following– the object starts at time 0, at some initial height Theta0, …
Probability – L15.8 Trajectory Estimation Illustration Read More »