Probability Theory for Dummies

Before we start examining the Monty Hall problem in any kind of detail, let's look at some probability notation and theory. This entry is intended for those readers who aren't already altogether familiar with basic probability theory. Those who are may read this as a review, or skip reading today's post.

I believe that about 50% of my readers are going to read this specific entry. The others may not be too interested in this particular topic, they may be deterred by the mathematical notation (don't let that deter you, though), or they may already be very familiar with the topics in this entry. However, taking a random reader, the probability that he'll read this entry, given that my assumption is correct, is 50%. Put another way:

P(Read) = 0.5

"P" is the notation for probability. "Read" is the event that he reads the entry, and 0.5 is the decimal form of the probability of that event. In English, P(Read) = 0.5 reads "the probability of the event that he reads this entry equals 50%".

Conditional probability (2)

However, I feel that at least 75% of my math-inclined readers probably are familiar with those topics and, thus, will skip reading this entry. So,

P(NOT Read | Math-inclined) = 0.75

"P" and "Read" are the same as above."NOT" is an operator stating that the "Read" event is not true. "|" is the the operator of conditional probability, and it reads "given". "Math-inclined" is the event that the chosen reader is one of those math-inclined readers. So this whole statement reads "The probability of the event that he does not read this entry, given the event that he's math-inclined, equals 75%".

The NOT operator

The chances that an event takes place, plus the chance that it does NOT, always adds up to 100%. Put differently, in 100% of cases, either the event takes place or it doesn't. So,

P(A) + P(NOT A) = 1.

With some very basic algebra, this can be turned into

P(NOT A) = 1 - P(A)

The probability that the math-inclined reader will read this entry equals 1 minus the probability that he will NOT read it, so

P(Read | Math-inclined) = 1 - P(NOT Read | Math-inclined) = 1 - 0.75 = 0.25

Joint probability

I believe that about 75% of my readers are math-inclined, so

P(Math-inclined AND Read) = 0.75 * 0.25 = 0.1875

There's an 18.75% chance that a randomly chosen reader is math-inclined but still reads this entry. "AND" is the operator of joint probability. This means that, for the statement "Math inclined AND Read" to be true, both events have to be true. The reason why we multiply the probabilities of both events can be illustrated as follows:

P(Math-inclined) = 0.75 = 3/4
P(Read | Math-inclined) = 0.25 = 1/4

as stated above. Out of 16 randomly picked readers, on average 12 readers will be math-inclined (3/4 * 16). Out of those 12 math-inclined readers, 3 will read this entry (1/4 * 12). So out of 16 initially chosen readers, 3 are math-inclined AND will read this entry. 3 out of 16 are 18.75% (3/16 = 0.1875). This is consistent with multiplying the probabilities of both required events: 3/4 * 1/4 = 3/16.

Put differently, in 75% of cases the randomly chosen reader is math-inclined. In 25% of those cases, he'll still read this entry. As we all know, a certain percentage of a quantity equals that quantity multiplied by the decimal form of the percentage. So, 25% of 75% is 0.25 * 75%, which equals 18.75% (0.1875).

The OR operator and the Sieve principle

P(Math-inclined OR NOT Math-inclined) = 0.75 + 0.25 = 1

Being put this way, this one seems pretty obvious. The probability that a reader is either math-inclined or not is obviously 1. But it may not seem equally obvious if we put it in more abstract terms. Formally,

P(A OR B) = P(A) + P(B) - P(A AND B)

Why is this? Well, let's resort to an intuitive explanation. Let's say we have six two-letter combinations, for example [AB, AC, AD, BC, BD, CD]. If we choose one of those combinations at random,

P(A) = 3/6 (the probability of the event that the letter "A" is represented in the combination)
P(B) = 3/6 (the probability of the event that the letter "B" is represented in the combination)

What's the probability that either A or B are represented in the combination? Well, from a quick look, we can see that it's 5/6, since only 1 in 6 combinations include neither A or B. But how do we arrive at this mathematically?

Adding the number of combinations where A is represented to the number of combinations where B is represented seems like a good first step. But this leaves us with 6/6, which is clearly wrong. This is because, in this way, we've double-counted the instance where both A and B are represented. So, having counted that instance twice, we need to subtract it once:

P(A OR B) = P(A) + P(B) - P(A AND B) = 3/6 + 3/6 - 1/6 = 5/6

This is called the Sieve principle, or the inclusion-exclusion principle. With more events, such as P(A OR B OR C) it gets a little bit trickier, but let's not bother with that right now.

Returning to

P(Math-inclined OR NOT Math-inclined) = 0.75 + 0.25 = 1

didn't I forget to subtract the double-counted instances? No. Double counting isn't possible in this case, since one reader can't be both math-inclined and non-math-inclined. So

P(Math-inclined AND NOT Math-inclined) = 0

and so, actually,

P(Math-inclined OR NOT Math-inclined) = 0.75 + 0.25 - 0 = 1

but we can just skip that step.

That's a very brief look at probability theory. We'll expand on it later on, and we'll use it when examining the Monty Hall problem further. But first, we'll have a more intuitive look at the Monty Hall problem tomorrow.

Next entry on this topic: Conditional Probability

(1) This, and all of the other figures concerning the inclinations and tendencies of my readers, are completely made up. They are probably not even nearly accurate, and I use them solely for illustration purposes.
(2) In a future entry, we'll have a more detailed look at conditional probability


Anonymous said...

Dafuq did i just read : |

Anonymous said...

Thank you for your post using a simple way to introduce probability theory. I need to understand basics to read research papers, thanks!

Post a Comment