Bayes theorem can be considered as the most important application of conditional probability. Conditional probability is defined considering two events that can occur together. Bayes' theorem extends this to a scenario where an event can occur with any one of a number of mutually exclusive events.
Suppose there are two baskets both containing apples and oranges. Let the first basket contains 10 apples and 12 oranges and in the second basket there are 15 apples and 12 oranges. Conditional probability problems deal with finding the probability of picking a type of fruit (apple or orange) when it is known which basket it has come from. Baye's Theorem deals with the inverse probability of the fruit coming from a basket when the type of the fruit is known.

In Baye's model, the probabilities are calculated expressing the given event as a union several intersections the event makes with the mutually exclusive events.

Suppose an event B can occur in combination with one of the n mutually exclusive events, A1, A2, ......An, and P(B) > 0, then Bayes theorem provides a rule for calculating the conditional probability of the occurrence of one of the above n events Ar, given that B has already occurred.
P(Ar | B) = $\frac{P(A_{r}\cap B)}{P(A_{1}\cap B)+P(A_{2}\cap B)+.....P(A_{n}\cap B)}$
Bayes theorem can be rewritten using the multiplication rule for probabilities,

P(Ar ∩ B)= P(Ar) . P(B | Ar)

P(Ar | B) = $\frac{P(A_{r})P(B|A_{r})}{P(A_{1})P(B|A_{1})+P(A_{2})P(B|A_{2})+.......P(A_{n})P(B|A_{n})}$
As a special case if the event B can occur with two mutually exclusive events A1 and A2, then the Baye's formulas are as follows:

P(A1 | B) = $\frac{P(A_{1})P(B|A_{1})}{P(A_{1})P(B|A_{1})+P(A_{2})P(B|A_{2})}$

and

P(A2 | B) = $\frac{P(A_{2})P(B|A_{2})}{P(A_{1})P(B|A_{1})+P(A_{2})P(B|A_{2})}$

The conditional probability of the event A, given that the event B has already occurred is

P(A | B) = $\frac{P(A\cap B)}{P(B)}$ ...................(1)

Suppose the event B can occur in combination with n mutually exclusive events A1, A2,......Ar,....An then
B = (A1 ∩ B) U (A2 ∩ B) U ......U (Ar ∩ B)...... U (An ∩ B)

Since the events (A1 ∩ B), (A2 ∩ B)......are mutually exclusive events applying the addition rule for probabilities

P(B) = P(A1 ∩ B) + P(A2 ∩ B) +......+ P(Ar ∩ B) + .........+ P(An ∩ B)
If in equation (1) A is replaced with Ar, then we get

P(Ar | B) = $\frac{P(A_{r}\cap B)}{P(A_{1}\cap B)+P(A_{2}\cap B)+.....P(A_{n}\cap B)}$
which is indeed the Bayes theorem on conditional probabilities.

Solved Examples

Question 1: For a hypothetical magazine, the probability that the reader is male given that the reader is at least 35 years old is 0.30. The probability that a reader is male, given that the reader is under 35, is 0.65. If 75% of the reader are under thirty five what is the probability that a randomly chosen reader is
     (a) male
     (b) female
     (c) under 35 if it is given the reader is a female
Solution:
 
Let us put the information given in the form of a table.

     (a)  Let A1 be the event of the reader being at least 35 years and A2 the event of the reader being under 35 years old
           and B the event of the reader being a male.
           P(A2) = 0.75   and P(A1) = 1 - P(A2) = 1- 0.75 = 0.25
           P(B | A1) = 0.30   and P(B | A2) = 0.65
           B = (A1 ∩ B) U (A2 ∩ B)   as the male reader can either under 35 or at least 35 years old.
           P(B) = P(A1 ∩ B) + P(A2 ∩ B)
                   = P(A1) . P(B | A1) + P(A2) . P(B | A2)
                   = (0.25)(0.30) + (0.75)(0.65) = 0.5625
           The probability the reader is male = 0.5625

     (b)  P(female) = 1 - P(male) = 1 - 0.5625 = 0.4375
           The probability the reader is female = 0.4375.

     (c) Let E1 be the event of the reader is under 35 and E2 the event of the reader is at least 35 and the F the event of the
          reader being a female.
         We need to find P( E1 | F) and we have the probabilities
         P(F | E1) = 1 - 0.65 = 0.35  , P(F | E2) = 1 - 0.30 = 0.70,   P(E1) = 0.75 and P(E2) = 0.25

         P( E1 | F)  = $\frac{P(E_{1})P(F|E_{1})}{P(E_{1})P(F|E_{1})+P(E_{2})P(F|E_{2})}$

                          = $\frac{(0.75)(0.35)}{(0.75)(0.35)+(0.25)(0.70)}$

                          = 0.6
        Probability the reader is under 35 given that the reader is a female = 0.6.
 

Question 2: The probability the event A will occur is 0.4 The conditional probability of another event B occurring given A is 0.6 and the conditional probability of B occurring given A has not occurred is 0.3. Find the probability of
     (a) the event A occurring given B
     (b) the event of A not occurring given B
Solution:
 
Expressing the given probabilities using notation
      P(A) = 0.4.  Hence P(Ac) = 1 - 0.4 = 0.6
      P(B | A) = 0.6   and P(B | Ac) = 0.3
      (a) We need to compute P(A | B)
            Using Baye's Theorem

            P(A | B) = $\frac{P(A)P(B|A)}{P(A)P(B|A)+P(A^{c})P(B|A^{c})}$

                         = $\frac{(0.4)(0.6)}{(0.4)(0.6)+(0.6)(0.3)}$

                         = 0.571
            Probability of event A occurring given B = 0.571
     (b) To find P(Ac | B)
           P(Ac | B) = 1 - P(A | B)
                         = 1 - 0.571 = 0.429
          Probability of event A not occurring given B + 0.429.
 


Practice Problems

Question 1: There are three urns containing Black, Red and Green Balls. Urn 1 contains 4 Black , 5 Red and 3 Green Balls. Urn 2 contains 7 Black , 4 Red and 4 Green balls. Urn 3 contains 6 Black, 6 Red and 5 Green Balls. A Urn is chosen and a ball is picked. If the ball picked is Green find the probability it has come from Urn 3.
      (Answer: 0.363)
Question 2: The probability of the incidence of a rare disease is 0.002. The probabilities of diagnostic disease showing positive are 0.97 and 0.05 for an infected and not an infected situations. When the test turns out positive, what is the probability the person tested is really has the disease.
    (Answer: 0.0374)