Conditional Probability

In my last entry on probability theory, I promised to have a more detailed look at conditional probability. We need this for solving the disease test problem and for solving the Monty Hall problem mathematically.

P(A | B) reads "the probability of A given B", and this is referred to as conditional probability. The formula for calculating this probability is

P(A | B) = P(A AND B) / P(B)

Why?

We're trying to figure out the probability that the event A is also true, given that B is true. Sometimes A may be true even though B is not, but we're not interested in those instances.

Independent events

Now, sometimes the probability of A is independent of B. For example, if we flip two coins [A, B], and each event A and B is true if the corresponding coin comes up heads. If B is true (that is, comes up heads) the probability of A is still 50% (1). This means that

P(A | B) = P(A)

where P(A) is the a-priori probability and the conditional probability is unchanged due to the information that we gained from flipping the coin B.

Dependent events

But what if

P(A | B) != P(A)

("!=" reads "does not equal") In this case, A is dependent on B. What this means is that, as we gain information about B, the probability of A changes from the a-priori probability. In this case, we need to consider all the cases when B is true:

P(A AND B) + P(NOT A AND B) = P(B)

Those are all of the instances where B is true. So we know that B is true. Out of all the instances where B is true [P(B)], some of them are instances where A is also true [P(A AND B)]:

P(A | B) = P(A AND B) / P(B)

An example

Let's say we have a drawn three cards from a deck: An ace, a king and a queen [A, K, Q]. We shuffle those three cards and draw two of them, trying to draw an ace. Let's say that the first card is not an A. What's the probability that the second one is?

First, let's define two events:

Card1 is the event that the first card is an A
Card2 is the event that the second card is an A

P(Card1) = 1/3
P(NOT Card1) = 2/3
P(Card2 AND NOT Card1) = 1/3

The last probability is easy to see from an a-priori standpoint - The probability that any one of the drawn cards will be an A is 1/3. So the probability of that one card to be an A and the other one not to be an A is also 1/3.

P(Card2 | NOT Card1) = P(Card2 AND NOT Card1) / P(NOT Card1)
P(Card2 | NOT Card1) = (1/3) / (2/3) = 1/2

More intuitively, this can be illustrated as follows:








Card 2

AKQ
Card 1A01/61/6
K1/601/6
Q1/61/60

As we can see in this matrix, there are six possible combinations of two cards. [AA, KK, QQ] are not possible, since there's only one card of each rank. Each possible combination has a 1/6 probability of occuring. [AK, AQ] are the possible combinations where the first card is an A. In the matrix, we can find the probabilities stated earlier:

P(Card1) = 2 * 1/6 = 1/3
P(NOT Card1) = 4 * 1/6 = 2/3
P(Card2 AND NOT Card1) = 2 * 1/6 = 1/3

Asking what P(Card2 | NOT Card1) is, is the same as asking "how big a fraction of the times that we don't pick an A as our first card do we pick an ace as our second card?". We can easily see in the matrix that there are 4 cases (marked as green) where we don't pick an A as our first card. In two of those cases our second card is an A. 2/4 = 1/2. But also, 2*(1/6) / 4*(1/6) = 1/2.

2*(1/6) / 4*(1/6) = P(Card2 AND NOT Card1) / P(NOT Card1)

Stated in words, in 2 out of 4 cases when the first card is not an A (out of a total of 6 possible cases, which includes draws where the first card is an A), the second card is an A. So, knowing that the first card is not an A, we can narrow the situation down to those 4 cases, giving us a probability of 2/4 = 1/2.

I hope this entry has helped your understanding of how conditional probability works. It's not very formal, and it's not very extensive, but hopefully it's quite intuitive and at least free of too big gaps in it's logic. However, it's late now, and I kind of just threw this one out there, because I haven't posted anything for a while.

__________
Notes:
(1) If you think otherwise, you're subjected to the gamblers fallacy, which we'll have a closer look at in a future post.

1 comments:

Dracothraxus said...

One of the better explanations out there. You have a simple typo (which you correct on the next line):

"P(Card2 | NOT Card1) = P(Card2 AND NOT Card1) / P(NOT Card2)"

should be

"P(Card2 | NOT Card1) = P(Card2 AND NOT Card1) / P(NOT Card1)"

Post a Comment