NBAYES_LNK(1)

NAME
nbayes_lnk - Naive Bayes Classifier

SYNOPSIS
Train and test a Naive Bayes classifier that models the likelihoods of categorical input features assuming feature independence.

nbayes_lnk [ -nbins n ] [ -nBinsPerInput n[,n]... ]
[ -discrete ] [ -range_factor x ] [ -ignore_unseen ]
[ -unity_laplace ] [ -classifier_options ]

EXAMPLE
nbayes_lnk -nbins 4 -train -create \
-pathdata $LNKHOME/data/class -finput vowel.train \
-fdescribe vowel.defaults -fparam nbayes.param \
-range_factor 1.2

nbayes_lnk -pathdata $LNKHOME/data/class \
-finput vowel.eval -fdescribe vowel.defaults \
-fparam nbayes.param

DESCRIPTION
This classifier is for problems where feature inputs can be made categorical and assigned to integer values starting at zero. The classifier estimates the likelihood of each class by computing the likelihood of each feature separately using counts from the training data and then multiplying feature likelihoods assuming features are independent given the class. The likelihood assigned to each feature is proportional to the number of training patterns that fall in a bin divided by the number of patterns in that class. In testing, these likelihoods are multiplied across input features to give an overall likelihood for each class. The naive Bayes classifier is different than the histogram classifier because it is explicitly designed to handle categorical data as described below. It is not necessarily naive or simple. A straightforward approach is used but it often provides good performance that rivals that of more complex classifiers. This classifier has proven especially useful with categorical features. Categorical features take on values that are not ordered in a meaningful way. An example would be an input feature used to classify Internet web servers that was the name of the web server host computer operating system. If there are 12 different types of operating systems, then this input feature would take on 12 values. For use with LNKnet, categorical features must be preprocessed and be assigned to integers ranging from zero to nvalues-1, where nvalues is the number of different values for the input feature. For the operating system example, the operating sytem input feature values range from 0 to 11. Note that input features must be preprocessed to take on these integer values and they should not be further normalized within LNKnet. For example, "simple normalization" as assigned in the LNKnet "Feature Normalization" window should not be used. This will change input feature values to be non-integers that do not range from 0 to nvalues-1. Likewise, other forms of normalization should not be used. Every implementation of naive Bayes classifiers must address three subtle issues. The first is how to assign probabilities to bins containing values not seen in any training patterns. The Laplace correction is used in this program because it often works well. A less common variant can be selected with the -unity_laplace flag described below. The second issue is how to assign bin probabilities for categorical features when training patterns take on values that are outside the expected range. For example if the number of bins for a feature is set to 12, then feature values should range from 0 to 11. Other input feature values such 12 or 21 are outside this range. This program creates an extra "unseen" bin for any feature where this occurs. All patterns that fall outside the expected range are counted as falling in this bin. The flag -ignore_unseen also allows such patterns to be ignored. The third issue is how to treat features in testing that take on values that are outside the expected range. This program ignores such features. If there are many features, and at least one takes on an expected value, but all others take on unexpected values, then classification is still possible and will be based on the one feature. If all features take on unexpected values, then no class will be selected and no classification decision will be made.

OPTIONS
-- Prints a list of all flags and arguments.

-nbins n
    The number of discrete input values for every input feature. If all features are binary and can take on two values (e.g. "yes" and "no"), then this flag should be used with an argument of 2.

-nBinsPerInput n[,n]...
    The argument gives the number of discrete values for each input feature. Note that these are for the list of features as presented to the classifier assuming no feature normalization but possibly with feature selection. If all the values of n are the same, then this has the same effect as the previous flag.

-discrete
    This must be set. It is a holdover from the histogram classifier that uses bins and means that bins are centered over numbers in the range (0,nvalues-1), taking the input features as indices into an array holding the class likelihoods for each bin. In this equation, nvalues is the number of different values the input feature can take on. With LNKnet, these values must be preprocessed to be integers ranging from 0 to nvalues-1. If the -discrete flag is set, -range_factor is ignored. In addition, if the -discrete flag is set, no type of LNKnet feature normalization should be used to pre-process features for this classifier. For example, "simple normalization" as assigned in the LNKnet "Feature Normalization" window will lead to non-integer feature values and should not be used.

-ignore_unseen
    This is a training flag which affects discrete classification. Normally in training, patterns with values that fall outside the number of allowed integer values for an input feature are counted in a single bin. That is, if the user has asked for bins ranging from 0 to 5 and a pattern comes in with a value of 6, the count for that pattern goes into the "unseen" bin for that feature. If ignore_unseen is set during training, the likelihood for the "unseen" bin is set to 1, making it have no effect during testing.

-unity_laplace
    This training flag changes the formula for the Laplace correction made to the likelihoods in each input bin. The normal formula is
         count[i,b,c] + 1/N
         -------------------
         Nc + (nbins[i]+1)/N
    where count[i,b,c] is the count of patterns in class c that fell into bin b for input feature i, Nc is the number of patterns for class c, N is the total number of patterns seen, and nbins[i] is the number of bins for input feature i. With unity_laplace set, the 1/N in the numerator and denominator is replaced by a 1, giving
         count[i,b,c] + 1
         ----------------
         Nc + (nbins[i]+1)

-classifier_options
    nbayes_lnk accepts the general classifier arguments described on the classifier(1) manual page.

    Notes about argments from general list:

    -random
        This flag has an effect only when adjust_train_priors is set.

SEE ALSO
classifier(1), nbayes2c(1), nbayes_feat_sel(1), nbayes_plot_bound(1)

HISTORY
First written 28 March 2001 by Linda Kukolich Modified 19 July 2002 by Richard Lippmann Modified 11 February 2004 by Richard Lippmann

BUGS
Send bug reports to Linda Kukolich (kukolich@sst.ll.mit.edu)