Bayes theorem can be considered as the most important application of conditional probability. Conditional probability is defined considering two events that can occur together. Bayes' theorem extends this to a scenario where an event can occur with any one of a number of mutually exclusive events.
Suppose there are two baskets both containing apples and oranges. Let the first basket contains 10 apples and 12 oranges and in the second basket there are 15 apples and 12 oranges. Conditional probability problems deal with finding the probability of picking a type of fruit (apple or orange) when it is known which basket it has come from. Baye's Theorem deals with the inverse probability of the fruit coming from a basket when the type of the fruit is known.

In Baye's model, the probabilities are calculated expressing the given event as a union several intersections the event makes with the mutually exclusive events.

Definition

Suppose an event B can occur in combination with one of the n mutually exclusive events, A1, A2, ......An, and P(B) > 0, then Bayes theorem provides a rule for calculating the conditional probability of the occurrence of one of the above n events Ar, given that B has already occurred.
P(Ar | B) = $\frac{P(A_{r}\cap B)}{P(A_{1}\cap B)+P(A_{2}\cap B)+.....P(A_{n}\cap B)}$

Formula

Bayes theorem can be rewritten using the multiplication rule for probabilities,

P(Ar ∩ B)= P(Ar) . P(B | Ar)

P(Ar | B) = $\frac{P(A_{r})P(B|A_{r})}{P(A_{1})P(B|A_{1})+P(A_{2})P(B|A_{2})+.......P(A_{n})P(B|A_{n})}$
As a special case if the event B can occur with two mutually exclusive events A1 and A2, then the Baye's formulas are as follows:

P(A1 | B) = $\frac{P(A_{1})P(B|A_{1})}{P(A_{1})P(B|A_{1})+P(A_{2})P(B|A_{2})}$

and

P(A2 | B) = $\frac{P(A_{2})P(B|A_{2})}{P(A_{1})P(B|A_{1})+P(A_{2})P(B|A_{2})}$

Proof

The conditional probability of the event A, given that the event B has already occurred is

P(A | B) = $\frac{P(A\cap B)}{P(B)}$ ...................(1)

Suppose the event B can occur in combination with n mutually exclusive events A1, A2,......Ar,....An then
B = (A1 ∩ B) U (A2 ∩ B) U ......U (Ar ∩ B)...... U (An ∩ B)

Since the events (A1 ∩ B), (A2 ∩ B)......are mutually exclusive events applying the addition rule for probabilities

P(B) = P(A1 ∩ B) + P(A2 ∩ B) +......+ P(Ar ∩ B) + .........+ P(An ∩ B)
If in equation (1) A is replaced with Ar, then we get

P(Ar | B) = $\frac{P(A_{r}\cap B)}{P(A_{1}\cap B)+P(A_{2}\cap B)+.....P(A_{n}\cap B)}$
which is indeed the Bayes theorem on conditional probabilities.

Examples

Solved Examples

Question 1: For a hypothetical magazine, the probability that the reader is male given that the reader is at least 35 years old is 0.30. The probability that a reader is male, given that the reader is under 35, is 0.65. If 75% of the reader are under thirty five what is the probability that a randomly chosen reader is
(a) male
(b) female
(c) under 35 if it is given the reader is a female
Solution:

Let us put the information given in the form of a table.

(a)  Let A1 be the event of the reader being at least 35 years and A2 the event of the reader being under 35 years old
and B the event of the reader being a male.
P(A2) = 0.75   and P(A1) = 1 - P(A2) = 1- 0.75 = 0.25
P(B | A1) = 0.30   and P(B | A2) = 0.65
B = (A1 ∩ B) U (A2 ∩ B)   as the male reader can either under 35 or at least 35 years old.
P(B) = P(A1 ∩ B) + P(A2 ∩ B)
= P(A1) . P(B | A1) + P(A2) . P(B | A2)
= (0.25)(0.30) + (0.75)(0.65) = 0.5625
The probability the reader is male = 0.5625

(b)  P(female) = 1 - P(male) = 1 - 0.5625 = 0.4375
The probability the reader is female = 0.4375.

(c) Let E1 be the event of the reader is under 35 and E2 the event of the reader is at least 35 and the F the event of the
We need to find P( E1 | F) and we have the probabilities
P(F | E1) = 1 - 0.65 = 0.35  , P(F | E2) = 1 - 0.30 = 0.70,   P(E1) = 0.75 and P(E2) = 0.25

P( E1 | F)  = $\frac{P(E_{1})P(F|E_{1})}{P(E_{1})P(F|E_{1})+P(E_{2})P(F|E_{2})}$

= $\frac{(0.75)(0.35)}{(0.75)(0.35)+(0.25)(0.70)}$

= 0.6
Probability the reader is under 35 given that the reader is a female = 0.6.

Question 2: The probability the event A will occur is 0.4 The conditional probability of another event B occurring given A is 0.6 and the conditional probability of B occurring given A has not occurred is 0.3. Find the probability of
(a) the event A occurring given B
(b) the event of A not occurring given B
Solution:

Expressing the given probabilities using notation
P(A) = 0.4.  Hence P(Ac) = 1 - 0.4 = 0.6
P(B | A) = 0.6   and P(B | Ac) = 0.3
(a) We need to compute P(A | B)
Using Baye's Theorem

P(A | B) = $\frac{P(A)P(B|A)}{P(A)P(B|A)+P(A^{c})P(B|A^{c})}$

= $\frac{(0.4)(0.6)}{(0.4)(0.6)+(0.6)(0.3)}$

= 0.571
Probability of event A occurring given B = 0.571
(b) To find P(Ac | B)
P(Ac | B) = 1 - P(A | B)
= 1 - 0.571 = 0.429
Probability of event A not occurring given B + 0.429.

Problems

Practice Problems

Question 1: There are three urns containing Black, Red and Green Balls. Urn 1 contains 4 Black , 5 Red and 3 Green Balls. Urn 2 contains 7 Black , 4 Red and 4 Green balls. Urn 3 contains 6 Black, 6 Red and 5 Green Balls. A Urn is chosen and a ball is picked. If the ball picked is Green find the probability it has come from Urn 3.