In this segment, we will discuss the multinomial model and the multinomial probabilities, which are a nice generalization of the binomial probabilities.
The setting is as follows.
We are dealing with balls and the balls come into different colors.
There are r possible different colors.
We pick a ball at random, and when we do that, there is a certain probability Pi that the ball that we picked has ith color.
Now, we repeat this process n times independently.
Each time we get a ball that has a random color.
And we’re interested in the following kind of question.
Somebody fixes for us certain numbers– n1, n2, up to nr that add up to n, and asks us, what is the probability that when you carry out the experiment, you get exactly n1 balls of the first color, exactly n2 balls of the second color, and so on? So the numbers n1, n2, up to nr are fixed given numbers.
For a particular choice of those numbers, we want to calculate this probability.
Now of course, this is a more general model.
It doesn’t necessarily deal with balls of different colors.
For example, we might have an experiment that gives us random numbers, where the numbers range from 1 up to r, and at each time we get a random number with probability Pi we get a number which is equal to i.
So we could use this to model die rolls, for example.
And there’s actually a special case of this problem, which should be familiar.
Suppose that we have only two colors, and instead of thinking of colors, let us think of the two possibilities as being heads or tails.
And we can make the following analogy.
Somebody gives us numbers n1 and n2 that add up to n.
And we’re interested in the probability that we get n1 of the first color and n2 of the second color.
Well, we could think of this as a setting in which we are asking for the probability that we obtain k heads and n minus k tails.
So the question of what is the probability that we obtain k heads and n minus k tails is of the same kind as what is the probability that we get n1 of the first color and n2 of the second color.
Now, if heads have a probability p of occurring, and tails has a probability of 1 minus p of occurring, then we would have the following analogy.
The probability of obtaining the first color, which correspond to heads, that would be equal to p.
The probability of obtaining the second color, which correspond to tails, this would be 1 minus p.
Now, the probability of obtaining k heads in those n independent trials– we know what it is.
By the binomial probabilities, it is n choose k times p to the k times one minus p to the power n minus k.
Now we can translate this answer to the multinomial case where we’re dealing with colors, and we do these substitutions.
So n choose k is n factorial divided by k factorial.
In this case, k is the same as n1, so we get n1 factorial.
And then we are going to have here n minus k factorial.
But n minus k corresponds to n2.
So here we get an n2 factorial.
And then p corresponds to p1 and p2 corresponds to 1 minus p.
So we get here p1 times p2.
n to the power n minus k, again, by analogy, is n2.
So this is the form of the multinomial probabilities for the special case where we’re dealing with two colors.
Let us now look at the general case.
Let us start with an example, to be concrete.
Suppose that the number of colors is equal to 3, and that we’re going to pick n equal to 7 balls.
We carry out the experiments, and we might obtain an outcome which would be a sequence of this type.
So the first ball had the color 1, the second ball had the first color, the third ball had the third color, the fourth ball had the first color, and so on.
And suppose that this was the outcome.
One way of summarizing what happened in this outcome would be to say that we had four 1s, we had two 2s, and we had one 3.
We could say that this is the type of the outcome.
It’s of type 4, 2, 1– that is, we obtained four of the first color, two of the second color, and one of the third color.
This is one possible outcome.
What is the probability of obtaining this particular outcome? The probability of obtaining this particular outcome is, using independence, the probability that we obtain color 1 in the first trial, color 1 in the second trial, color 3 in the third trial, color 1 in the fourth trial, color 2 in the next trial, color 2 in the next trial, color 1 in the last trial.
And we put all the factors together, and we notice that this is p1 to the fourth p2 to the second times p3.
It’s not a coincidence that the exponents that we have up here are exactly the count that we had when we specified the type of this particular outcome.
Generalizing from this example, we realize that the probability of obtaining a particular sequence of a certain type, that probability is of this form.
For each color, we have the probability of that color raised to the power of how many times that particular color appears in a sequence.
So any particular sequence of this type has this probability.
What we’re interested in is to find the total probability of obtaining some sequence of this type.
How can we find this probability? Well, we will take the probability of each sequence of this type– which is this much, and it’s the same for any particular sequence– and multiply with the number of sequences of this type.
So how many sequences are there of a certain type? Let us look back at our example.
We had seven trials.
So let us number here the different trials.
And when I tell you that a particular sequence was obtained, that’s the same as telling you that in this set of trials, we had the first color.
In this set of trials, the fifth and sixth trial, we had the second color.
And in this trial, the third trial, we had the third color.
This is an alternative way of telling you what sequence we obtained.
I tell you at which trials we had the first color, at which trials we had the second, at which trials we had the third.
But What do we have here? Here we have a partition of the set of numbers from 1 up to 7 into three subsets.
And the cardinalities of those subsets are the numbers that appear here in the type of the sequence.
The conclusion is that a sequence of certain type is equivalent, or can be alternatively specified, by giving you a partition over this set of tosses, which is the set from 1 up to n, how many trials we’ve had, a partition into subsets of certain sizes.
So this allows us now to count the number of sequences of a certain type.
It’s exactly the same as the number of partitions, and we know what this is.
And putting everything together, the probability of obtaining a sequence of a certain type is equal to the count of how many sequences do we have of the certain type, which is the same as the number of partitions of a certain type, times the probability of any particular sequence of that type that we’re interested in.
So this is a formula that generalizes the one that we saw before for the case where we have only two colors, and which corresponded to the coin tossing setting.
And it is a useful model, because you can think of many situations in which you have repeated trials, and at each trial, you obtain one out of a finite set of possible results.
There are different possible results.
You repeat those trials independently.
And you may be interested in the question of how many results of the first kind, of the second kind, and so on there will be.