In this segment we look into the probability that the sum of n independent identically distributed random variables takes an abnormally large value. We will get an upper bound on this quantity, which is known as Hoeffding’s inequality. This is an upper bound that applies to a special case, although the method actually generalizes. Here …
Let X be a random variable, and let g be a function. We know that if g is linear, then the expected value of the function is the same as that linear function of the expected value. On the other hand, we know that when g is nonlinear, in general, these two quantities will not …
This is a rather theoretical exercise that has two purposes. One is to verify that the notion of convergence in probability is quite natural and that it has properties similar to the notion of convergence of sequences. And the second purpose is to get a little bit of practice with the formal definition of convergence …
Probability – S18.1 Convergence in Probability of the Sum of Two Random Variables Read More »
The purpose of this segment is to give you a little bit of the bigger picture. We did discuss some inequalities, we did discuss convergence of the sample mean– that’s the weak law of large numbers– and we did discuss a particular notion of convergence of random variables, convergence in probability. How far can we …
We will now go through two examples of convergence in probability. Our first example is quite trivial. We’re dealing with a sequence of random variables Yn that are discrete. Most of the probability is concentrated at 0. But there is also a small probability of a large value. Because the bulk of the probability mass …
Probability – L18.7 Convergence in Probability Examples Read More »
We will now take a step towards abstraction, and discuss the issue of convergence of random variables. Let us look at the weak law of large numbers. It tells us that with high probability, the sample mean falls close to the true mean as n goes to infinity. We would like to interpret this statement …
We will now consider a very practical application of the weak law of large numbers, and the calculations associated with it. The application has to do with polling. There’s a certain referendum that’s going to take place. We’re close enough to the day of the referendum so that voters have made up their minds, and …
In this segment, we derive and discuss the weak law of large numbers. It is a rather simple result, but plays a central role within probability theory. The setting is as follows. We start with some probability distribution that has a certain mean and variance, which we assume to be finite. We then draw independent …
Probability – L18.4 The Weak Law of Large Numbers Read More »
Mathematically speaking, the Chebyshev inequality is just a simple application of the Markov inequality. However, it contains a somewhat different message. Consider a random variable that has a certain mean and variance. What the Chebyshev inequality says is that if the variance is small, then the random variable is unlikely to fall too far off …
In this segment, we derive and discuss the Markov inequality, a rather simple but quite useful and powerful fact about probability distributions. The basic idea behind the Markov inequality as well as many other inequalities and bounds in probability theory is the following. We may be interested in saying something about the probability of an …