Intro. Let us now illustrate the linear least mean squares estimation methodology in the context of an example. And we’re going to revisit our familiar example that we considered earlier in the context of general least mean squares estimation. Assumptions. Let us remind ourselves what were the assumptions behind this example. There is an unknown …
Now that we have found the solution to the linear least mean squares estimation problem, it is time to offer a few comments, make some observations, and provide some insights. A first important observation is the following. In order to implement this estimator, you do not really need to know everything about the distribution of …
Probability – L17.4 Remarks on the LLMS Solution and on the Error Variance Read More »
We will now develop the solution to the linear least mean squares estimation problem. We want to find coefficients a and b that make this expression, this mean squared error, as small as possible. We will approach this problem by proceeding in two stages. In the first stage, we assume that the choice of a …
Probability – L17.3 Solution to the LLMS Problem Read More »
Let us now introduce the linear least mean squares formulation. The setting is the usual one– we have an unknown random variable and another random variable, which is our observation. We’re given enough information so that we can, for example, calculate the joint distribution of these two random variables. What we would like to do …
If our objective is to keep the mean squared estimation error small, then the best possible estimator is the conditional expectation. But sometimes the conditional expectation is hard to calculate. Maybe we’re missing the details of the various probability distributions. Or maybe we have the distributions that we need but the formulas are complicated. After …
In this segment, we’re going to go over a few theoretical properties of the estimation error in least mean squares estimation. Recall that our least mean squares estimator is the conditional expectation of the unknown random variable, given our observations. Let us define the error, which is the difference between the estimator and the random …
Probability – L16.8 Properties of the LMS Estimation Error Read More »
Our discussion of least mean squares estimation so far was based on the case where we have a single unknown random variable and a single observation. And we’re interested in a point estimate of this single unknown random variable. What happens if we have multiple observations or parameters? For example, suppose that instead of a …
Probability – L16.7 LMS Estimation with Multiple Observations or Unknowns Read More »
We now continue with our example and turn to the performance evaluation question. As you recall, we have a Theta that has a certain prior distribution. We’re given a model for the observations. We came up with the joint distribution for X and Theta, which was uniform on this particular shape, and we found that …
Probability – L16.6 Example Continued: LMS Performance Evaluation Read More »
Let us now go through an example. Suppose that we have an unknown random variable Theta that has a uniform distribution between 4 and 10. We observe some other random variable X that’s related to Theta according to the following model. This is the conditional distribution of X given Theta. For any given value of …
Least mean squares estimation is remarkable because it has such a simple answer. The way to come up with estimates, if what you care about is to keep the mean squared error small, the way to come up with estimates is to just report the conditional expectation, which is going to be a number, once …