SAP HANA Predictive Analysis Library (PAL)

3.2.10 Naive Bayes
Naive Bayes is a classification algorithm based on Bayes theorem. It estimates the class-conditional
probability by assuming that the attributes are conditionally independent of one another.
Given the class label y and a dependent feature vector x 1 through x n , the conditional independence
assumption can be formally stated as follows:
Using the naive independence assumption that
P(x i |y, x 1 , ..., x i-1 , x i+1 , ..., x n ) = P(x i |y)
for all i, this relationship is simplified to
Since P(x 1 , ..., x n ) is constant given the input, we can use the following classification rule:
We can use Maximum a posteriori (MAP) estimation to estimate P(y) and P(x i |y). The former is then the
relative frequency of class y in the training set.
The different Naive Bayes classifiers differ mainly by the assumptions they make regarding the distribution
of P(x i |y).
For continuous attributes, the attribute data are fitted to a Gaussian distribution and get the P(x i |y).
For discrete attributes, the count number ratio is used as P(x i |y). However, if there are categories that did
not occur in the training set, P(x i |y) will become 0, while the actual probability is merely small instead of 0.
This will bring errors to the prediction. To handle this issue, PAL introduces Laplace smoothing. The P(x i |y) is
then denoted as:
This is a type of shrinkage estimator, as the resulting estimate is between the empirical estimate xi / N, and
the uniform probability 1/d. α > 0 is the smoothing parameter, also called Laplace control value in the
following discussion.
Despite its simplicity, Naive Bayes works quite well in areas like document classification and spam filtering,
and it only requires a small amount of training data to estimate the parameters necessary for classification.
The Naive Bayes algorithm in PAL includes two functions: NBCTRAIN for generating training model; and
NBCPREDICT for making prediction based on the training model.
SAP HANA Predictive Analysis Library (PAL)
PAL Functions P U B L I C 193