We will now take a step towards abstraction, and discuss the issue of convergence of random variables. Let us look at the weak law of large numbers. It tells us that with high probability, the sample mean falls close to the true mean as n goes to infinity. We would like to interpret this statement …
We will now consider a very practical application of the weak law of large numbers, and the calculations associated with it. The application has to do with polling. There’s a certain referendum that’s going to take place. We’re close enough to the day of the referendum so that voters have made up their minds, and …
In this segment, we derive and discuss the weak law of large numbers. It is a rather simple result, but plays a central role within probability theory. The setting is as follows. We start with some probability distribution that has a certain mean and variance, which we assume to be finite. We then draw independent …
Probability – L18.4 The Weak Law of Large Numbers Read More »
Mathematically speaking, the Chebyshev inequality is just a simple application of the Markov inequality. However, it contains a somewhat different message. Consider a random variable that has a certain mean and variance. What the Chebyshev inequality says is that if the variance is small, then the random variable is unlikely to fall too far off …
In this segment, we derive and discuss the Markov inequality, a rather simple but quite useful and powerful fact about probability distributions. The basic idea behind the Markov inequality as well as many other inequalities and bounds in probability theory is the following. We may be interested in saying something about the probability of an …
In this lecture, we develop the weak law of large numbers. Loosely speaking, the weak law of large numbers says that if we have a sequence of independent random variables with the same distribution, then the average of these random variables, which is called the sample mean, approaches the expected value of the distribution. In …
Introduction. In this final segment, we want to discuss an interesting point about linear estimators. Here’s what the issue is. You obtain an observation, X, on the basis of which you want to estimate Theta. But perhaps you measure X on a different scale, let’s say on a cubic scale, so that what you record …
Probability – L17.9 The Representation of the Data Matters in LLMS Read More »
Introduction. As we discussed earlier– even if we have multiple observations we can still find the structure of the best linear estimator in a fairly simple, computational way– by solving a system of linear equations. But usually, we do not get nice and simple formulas. But here is a nice example, in which we will …
Probability – L17.8 The Simplest LLMS Example with Multiple Observations Read More »
We have presented the complete solution to the liner least mean squares estimation problem, when we want to estimate a certain unknown random variable on the basis of a different random variable X that we get to observe. But what if we have multiple observations? What would be the analogous formulation of the problem? Here’s …
Probability – L17.7 LLMS with Multiple Observations Read More »
Let’s now go through another example, which will be a little more challenging. We’re going to revisit an old problem. We have a coin that has an unknown bias, Theta. And we have a prior distribution on this Theta. We fix some positive integer, n, we flip a coin n times, that has this unknown …
Probability – L17.6 LLMS for Inferring the Parameter of a Coin Read More »