We now come to the third and final kind of calculation out of the calculations that we carried out in our earlier example.
The setting is exactly the same as in our discussion of the total probability theorem.
We have a sample space which is partitioned into a number of disjoint subsets or events which we think of as scenarios.
We’re given the probability of each scenario.
And we think of these probabilities as being some kind of initial beliefs.
They capture how likely we believe each scenario to be.
Now, under each scenario, we also have the probability that an event of interest, event B, will occur.
Then the probabilistic experiment is carried out.
And perhaps we observe that event B did indeed occur.
Once that happens, maybe this should cause us to revise our beliefs about the likelihood of the different scenarios.
Having observed that B occurred, perhaps certain scenarios are more likely than others.
How do we revise our beliefs? By calculating conditional probabilities.
And how do we calculate conditional probabilities? We start from the definition of conditional probabilities.
The probability of one event given another is the probability that both events occur divided by the probability of the conditioning event.
How do we continue? We simply realize that the numerator is what we can calculate using the multiplication rule.
And the denominator is exactly what we calculate using the total probability theorem.
So we have everything we need to calculate those revised beliefs, or conditional probabilities.
And this all there is in the Bayes rule.
It is actually a very simple calculation.
It’s a very simple calculation.
However, it is a quite important one.
Its history goes way back.
In the middle of the 18th century, a Presbyterian minister, Thomas Bayes, worked it out.
It was published a few years after his death.
And it was quickly reorganized for its significance.
It’s a systematic way for incorporating new evidence.
It’s a systematic way for learning from experience.
And it forms the foundation of a major branch of mathematics, so-called Bayesian inference, which we will study in some detail later in this course.
The general idea is that we start with a probabilistic model, which involves a number of possible scenarios.
And we have some initial beliefs on the likelihood of each possible scenario.
There’s also some particular event that may occur under each scenario.
And we know how likely it is to occur under each scenario.
This is our model of the situation.
Under each particular situation, the model tells us how likely event B is to occur.
If we actually observe that B occurred, then we use that information to draw conclusions about the possible causes of B, or conclusions about the more likely or less likely scenarios that may have caused this events to occur.
That’s what inference is.
Having observed b, we make inferences as to how likely a particular scenario, Ai, is going to be.
And that likelihood is captured by this conditional probabilities of Ai, given the event B.
So that’s what the Bayes rule is doing.
Starting from conditional probabilities going in one direction, it allows us to calculate conditional probabilities going in the opposite direction.
It allows us to revise the probabilities of the different scenarios, taking into account the new information.
And that’s exactly what inference is all about, as we’re going to see later in this class.