We will now continue and derive some additional properties of probability laws which are, again, consequences of the axioms that we have introduced.
The first property is the following. If we have two sets and one set is smaller than the other– so we have a picture as follows. We have our sample space. And we have a certain set, A.
And then we have a certain set, B, which is even bigger. So the set B is the bigger blue set. So if B is a set which is larger than A, then, naturally, the probability that the outcome falls inside B should be at least as big as the probability that the outcome falls inside A. How do we prove this formally?
The set B can be expressed as a union of two pieces. One piece is the set A itself. The second piece is whatever elements of B there are, that do not belong in A. What are these elements?
They are elements that belong to B. And they do not belong to A, which means that they belong to the complement of A. So we have expressed the set B as the union of two pieces. Now this piece is A. This piece here is outside A. So these two pieces are disjoint.
And so we can apply the additivity axiom, and write that the probability of B is equal to the probability of A plus the probability of the other set. And since probabilities are non-negative, this expression here is at least as large as the probability of A. And this concludes the proof of the property that we wanted to show.
Indeed, the probability of A is less than or equal to the probability of B. The next property we will show is the following. It allows us to write the probability of the union of two sets for the case now, where the two sets are not necessarily disjoint.
So the picture is as follows. We have our two sets, A and B. These sets are not necessarily disjoint. And we want to say something about the probability of the union of A and B.
Now the union of A and B consists of three pieces. One piece is this one here. And that piece consists of those elements of A that do not belong to B. So they belong to B complement.
This set has a certain probability, let’s call it little a and indicate it on this diagram. So a is the probability of this piece. Another piece is this one here, which is the intersection of A and B. It has a certain probability that we denote by little b.
This is the probability of A intersection B. And finally, there’s another piece, which is out here. And that piece has a certain probability c. It is the probability of that set.
And what is that set?
That set is the following. It’s that part of B that consists of elements that do not belong in A. So it’s B intersection with the complement of A. Now let’s express the two sides of this equality here in terms of little a, little b, and little c, and see whether we get the same thing.
So the probability of A union B. A union B consists of these three pieces that have probabilities little a, little b, and little c, respectively.
And by the additivity axiom, the probability of the union of A and B is the sum of the probabilities of these three pieces. Let’s look now at the right hand side of that equation and see whether we get the same thing. The probability of A plus the probability of B, minus the probability of A intersection B is equal to the following. A consists of two pieces that have probabilities little a and little b.
The set B consists of two pieces that have probabilities little b and little c. And then we subtract the probability of the intersection, which is b. And we notice that we can cancel here one b with another b. And what we are left with is a plus b plus c. So this checks.
And indeed we have this equality here. We have verified that it is true. One particular consequence of the equality that we derived is the following.
Since this term here is always non-negative, this means that the probability of A union B is always less than or equal to the probability of A plus the probability of B. This inequality here is quite useful whenever we want to argue that a certain probability is smaller than something. And it has a name. It’s called the union bound.
We finally consider one last consequence of our axioms. And namely, we are going to derive an expression, a way of calculating the probability of the union of three sets, not necessarily disjoint. So we have our sample space.
And within the sample space there are three sets– set A, set B, and set C. We are going to use a set theoretic relation. We are going to express the union of these three sets as the union of three disjoint pieces. What are these disjoint pieces?
One piece is the set A itself. The second piece is going to be that part of B which is outside A. So this is the intersection of B with the complement of A. The third piece is going to be whatever is left in order to form the union of the three sets.
What is left is that part of C that does not belong to A and that does not belong to B. So that part is C intersection with A complement and B complement.
Now this set here, of course, is the same as that set because intersection of two sets is the same no matter in which order we take the two sets. And similarly, the set that we have here is the same one that appears in that expression.
Now we notice that these three pieces, the red, the blue, and the green, are disjoint from each other. So by the additivity axiom, the probability of this union here is going to be the sum of the probabilities of the three pieces.
And that’s exactly the expression the we have up here.