In this lecture, we will deal with a single topic. How to find the distribution, that is, the PMF or PDF of a random variable that is defined as a function of one or more other random variables with known distributions. Why is this useful? Quite often, we construct a model by first defining some …
We now look at an application of the Bayes rule that’s involves a continuous unknown random variable, which we try to estimate based on a discrete measurement. Our model will be as follows. We observe the discrete random variable K, which is Bernoulli, so it can take two values, 1 or 0. And it takes …
Probability – L10.11 Inference of the Bias of a Coin Read More »
We will now use the Bayes rule in an important application that involves a discrete unknown random variable and a continuous measurement. Our discrete unknown random variable will be one that takes the values plus or minus 1 with equal probability. And the measurement will be another random variable, Y, which is equal to the …
Probability – L10.10 Detection of a Binary Signal Read More »
We have seen two versions of the Bayes rule– one involving two discrete random variables, and another that involves two continuous random variables. But there are many situations in real life when one has to deal simultaneously with discrete and continuous random variables. For example, you may want to recover a discrete digital signal that …
If you remember our discussion from a long time ago, we said that much of this class consists of variations of a few basic skills and ideas, one of which is the Bayes rule, the foundation of inference. So let’s look here at the Bayes rule again and its different incarnations. In a discrete setting …
Just in order to get some more familiarity with joint PDFs, let us look at independent normals. Actually, this is an important example because noise is often modeled by normal random variables, and noise terms that show up at different parts of a system, or at different times, are often assumed to be independent. Suppose …
We will now go through an example that brings together all of the concepts that we have introduced. We have a stick of length l. And we break that stick at some random location, which corresponds to a random variable, X. And we assume that this random variable is uniform over the length of the …
Independence is one of the central concepts of probability theory, because it allows us to build large models from simpler ones. How should we define independence in the continuous case? Our guide comes from the discrete definition. By analogy with the discrete case, we will say that two jointly continuous random variables are independent if …
Conditional PDFs share most of the properties of conditional PMFs. All facts for the discrete case have continuous analogs. The intuition is more or less the same, although it is much easier to grasp in the discrete case. For example, we have seen this version of the total probability theorem. There is a continuous analog …
Probability – L10.4 Total Probability & Total Expectation Theorems Read More »
The definition of the conditional PDF is very simple. It is just this formula, which is analogous to the one for the discrete case. In all respects– mathematical and intuitive– it is very similar to the conditional PMF. Even so, developing a solid grasp of this concept does take some further thinking, so we will …
Probability – L10.3 Comments on Conditional PDFs Read More »