We now look at an application of the Bayes rule that’s involves a continuous unknown random variable, which we try to estimate based on a discrete measurement.
Our model will be as follows.
We observe the discrete random variable K, which is Bernoulli, so it can take two values, 1 or 0.
And it takes those values with probabilities y and 1 minus y, respectively.
This is our model of K.
The catch is that the value of y is not known.
And it is modeled as a random variable by itself.
You can think of a situation where we are dealing with a single coin flip.
We observe the outcome of the coin flip, but the coin is biased.
The probability of heads is some unknown number, y.
And we try to infer or say something about the bias of the coin on the basis of the observation that we have made.
So what do we assume about this y or the bias of the coin? If we know nothing about this random variable, we might as well model it as a uniform random variable on the unit interval.
And the question now is, given that we made one observation and the outcome was 1, what can we say about the probability distribution of Y given this particular information? So the question that we’re asking is, what we can tell about the density of Y given that the value of 1 has been observed.
The way to approach this problem is by using a version of the Bayes rule.
We want to calculate this quantity for the special case where k is equal to 1.
So let us calculate the various pieces on the right hand side of this equation.
The first piece is the density of Y.
This is the prior density before we obtain any measurement.
And since the random variable is uniform, this is equal to 1 for y in the unit interval.
And of course, it is 0 otherwise.
The next piece that we need is the distribution of K given the value of Y.
Well, given Y, K takes a value of 1, with probability equal to Y– so the probability of 1, if we’re told the value of y is just a y itself.
y is the bias of the coin that we’re dealing with.
The next term that we need is the denominator.
We will use this formula.
It is the integral of the density of Y, which is equal to 1.
And it is equal to 1 only on the range from 0 to 1, times this probability that K takes a value, a certain value.
In this case, we’re dealing with a value of 1, so here we’re going to put 1 instead of k.
And therefore, we’re dealing with this expression here, which is just y.
And we integrated over y’s.
So this is y squared over 2, evaluated at 0 and 1, which gives us 1/2.
So this is the unconditional probability that K is equal to 1.
If we know nothing about Y, by symmetry, higher biases are equally likely as lower biases.
So we should expect that it’s equally likely to give us a 1 as it is to give us a 0.
Now, we have in our hands all the pieces that go into this particular formula.
And we can go ahead with the final calculation.
So in the numerator, we have 1 times this term, evaluated at k equal to 1, which is equal to y.
And then in the denominator, we have a term that evaluates to 1/2.
So the final answer is 2y.
Over what range of y’s is this correct? Only for those y’s that are possible.
So this is for y’s in the unit interval.
If we are to plot this PDF, it has this shape.
This is a plot of the PDF of Y given that the random variable K takes on a value of 1.
Initially, we started with a uniform for Y.
So all values of Y were equally likely.
But once we observed an outcome of 1, this tells us that perhaps Y is on the higher end rather than lower end.
So after we obtain our observation, the random variable Y has this distribution, with higher values being more likely than lower values.
This example is a prototype of situations where we want to estimate a continuous random variable based on discrete measurements.
Essentially it is the same as trying to estimate the bias of a coin based on a single measurement of the result of a coin flip.
As you can imagine, there are generalizations in which we observe multiple coin flips.
And this is an example that we will see later on in this class.