In this segment, we derive and discuss the Markov inequality, a rather simple but quite useful and powerful fact about probability distributions. The basic idea behind the Markov inequality as well as many other inequalities and bounds in probability theory is the following. We may be interested in saying something about the probability of an […]

In this lecture, we develop the weak law of large numbers. Loosely speaking, the weak law of large numbers says that if we have a sequence of independent random variables with the same distribution, then the average of these random variables, which is called the sample mean, approaches the expected value of the distribution. In

Introduction. In this final segment, we want to discuss an interesting point about linear estimators. Here’s what the issue is. You obtain an observation, X, on the basis of which you want to estimate Theta. But perhaps you measure X on a different scale, let’s say on a cubic scale, so that what you record

Introduction. As we discussed earlier– even if we have multiple observations we can still find the structure of the best linear estimator in a fairly simple, computational way– by solving a system of linear equations. But usually, we do not get nice and simple formulas. But here is a nice example, in which we will

We have presented the complete solution to the liner least mean squares estimation problem, when we want to estimate a certain unknown random variable on the basis of a different random variable X that we get to observe. But what if we have multiple observations? What would be the analogous formulation of the problem? Here’s

Let’s now go through another example, which will be a little more challenging. We’re going to revisit an old problem. We have a coin that has an unknown bias, Theta. And we have a prior distribution on this Theta. We fix some positive integer, n, we flip a coin n times, that has this unknown

Intro. Let us now illustrate the linear least mean squares estimation methodology in the context of an example. And we’re going to revisit our familiar example that we considered earlier in the context of general least mean squares estimation. Assumptions. Let us remind ourselves what were the assumptions behind this example. There is an unknown

Now that we have found the solution to the linear least mean squares estimation problem, it is time to offer a few comments, make some observations, and provide some insights. A first important observation is the following. In order to implement this estimator, you do not really need to know everything about the distribution of

We will now develop the solution to the linear least mean squares estimation problem. We want to find coefficients a and b that make this expression, this mean squared error, as small as possible. We will approach this problem by proceeding in two stages. In the first stage, we assume that the choice of a

Let us now introduce the linear least mean squares formulation. The setting is the usual one– we have an unknown random variable and another random variable, which is our observation. We’re given enough information so that we can, for example, calculate the joint distribution of these two random variables. What we would like to do