Review of Probability Theory - Sorry

More documents

Recommendations

Info

To justify this procedure, let us calculate the probability of error whenever we make a decision. Whenever we observe a particular x, Clearly, in every instance in which we observe the same value for x, we can minimize the probability error by deciding: • if , and • if . Of course, we may never observe exactly the same value of x twice. Will this rule minimize the average probability of error? Yes, because the average probability of error is given by: and if for every x, P(error/ x) is as small as possible, the integral must be as small as possible. Thus, we have justified the following Bayes‘ decision rule for minimizing the probability of error: This form of the decision rule emphasizes the role of the a posteriori probabilities. By using equation 3, we can express the rule in terms of conditional and a priori probabilities. Note that p(x) in equation 3 is unimportant as far as making a decision is concerned. It is basically just a scale factor that assures us that By eliminating this scale factor, we obtain the following completely equivalent decision rule: .
Some additional insight can be obtained by considering a few special cases: • if for some x, p(x | 1 ) = p(x | 2 ) , then that particular observation gives us no information about the state of nature; in this case, the decision hinges entirely on the a priori probabilities. • On the other hand, if P( 1 ) = P( 2 ), then the states of nature are equally likely a priori; in this case the decision is based entirely on , p(x | j ), the likelihood of j with respect to x. In general, both of these factors are important in making a decision, and the Bayes‘ decision rule combines them to achieve the minimum probability of error. 3. Classifier Consider a set of elements. With Bayes' decision rule, it is possible to divide the elements of into p classes C 1 , C 2 , …, C p , from n discriminant attributes A 1 , A 2 , …, A n . We must already have examples of each class to choose typical values for the attributes of each class. The probability of meeting element ∈ , having attribute A i , given that we consider class C l , will be denoted by p Ai ( | C l ). If we put all these probabilities together for each attribute, we obtain the global probability of meeting element , given that the class is C l : (5) Classification must allow the class of unknown element to be decided with the lowest risk of error. Decision in Bayes' theory chooses the class C l for which the a posteriori membership probability p(C l | ). is the highest: (6) According to Bayes' rule, the a posteriori probability of membership p(C l | ) is calculated from the a priori probabilities of membership of element to class C l :
Page 1 and 2: http://www.survey.ntua.gr/main/labs
Page 3 and 4: 4. Bayes‘ Decision theory Bayes
Page 5: where (4) Bayes‘rule shows how th
Page 9: (8) With such an assumption, condit

Review of Probability Theory - Sorry

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?