classifier(1)

NAME
classifier - Common flags used in all LNKnet classifiers and clusterers.

SYNOPSIS
This manual page describes flags and functions common to all the classifiers and clusterers in the LNKnet package.

classifier [ -train ] [ -create ] [ -pathexp directory ]
[ -ferror file ] [ -fparam file ]
[ -pathdata directory ] -finput file
-fdescribe file
[ -npatterns n ] [ -ninputs n ]
[ -features n[,n]... ]
[ -normalize -fnorm file ]
[ -cross_valid n [ -random_cv ] ]
[ -cross_valid_from_file -fcross_valid file ]
[ -random ] [ -seed n ]
[ -adjust_train_priors [ -priors_npatterns n ]
[ -train_uniform_priors | -train_priors_list x[,x]... ] ]
[ -adjust_test_priors
[ -test_uniform_priors | -test_priors_list x[,x]... ] ]
[ -debug n ] [ -verbose n ] [ -verror n ]
[ -log_outputs ]

DESCRIPTION

All classification and clustering programs in LNKnet share a common user interface. This manual page explains the use of common flags and the basic operation of programs.

There are three major modes of operation for each classifier:
    training, testing, and cross validation testing.

    In training, the data is read in and presented to the classifier. If a
        classifier uses single pass training, all of the data is presented once. If a classifier does incremental training, all of the data is presented several times. Each complete presentation of the data is called an epoch. Incrementally trained classifiers do testing on the data while training and produce an error file and error summary like the ones produced during testing. When training is complete, parameters necessary to reconstruct this classifier are written to the parameter file.

        EXAMPLE: This command trains a K-Nearest Neighbor classifier using data in the file vowel.train and produces a parameter file X1knn.param: knn -train -create \
        -pathdata $LNKHOME/data/class \
        -finput vowel.train \
        -fdescribe vowel.defaults -fparam knn.param

    In testing, which is the default running mode of the classification
        programs, parameters necessary to construct a previously trained classifier are read in from the parameter file. Each pattern is presented to a classifier and pattern by pattern results are printed to an error file. A summary of the testing results is printed to the standard output. How much information is contained in that summary is controlled by the verbosity flag.

        Testing is not possible for the clustering programs.

        EXAMPLE: This command tests a K-Nearest Neighbor classifier trained previously using data in the file vowel.eval and parameters from the file knn.param: knn -pathdata $LNKHOME/data/class \
        -finput vowel.eval -fdescribe vowel.defaults \
        -fparam knn.param

    In N-fold cross validation testing, the training data is split into several
        parts, called folds. A classifier is trained and tested N times. Each time the classifier is trained on data from all but one left out fold and parameters are stored in the parameter file. This classifier is then immediately used for testing on the left out field and the results of the testing are written to the error file.

        Although clusterers cannot do cross validation testing, they can run in cross validation mode. The several sets of cluster centers saved in the parameter file are, in this case, read by a classifier when it is run in cross validation mode.

        EXAMPLE: This command tests a K-Nearest Neighbor classifier with 4-fold cross validation. The KNN classifier is trained and tested four times each time training on three folds of the training data and testing on one left out fold. knn -pathdata $LNKHOME/data/class \
        -finput vowel.train -fdescribe vowel.defaults \
        -cross_valid 4

    The  a priori
        class probability or prior probability is the assumed frequency of occurrence of different classes. Normally, classifiers assume priors are equal. Priors can be adjusted if desired by weighting outputs of classifiers with continuous outputs or by differentially sampling training data from different classes. Flags for priors adjustment must be consistent with one another.

        Training example:
        In the training example, the training data will be sampled to present an approximately equal number of patterns from each class to the Gaussian classifier even though there are ten times as many training patterns in the first as in the second class. Half of the patterns in the first class will be presented. The patterns in the second class will be presented five times each. gauss -train -create -adjust_train_priors \
        -priors_npatterns 1000 -train_uniform_priors \
        -pathdata $LNKHOME/data/class \
        -finput uniform_10_1.train \
        -fdescribe uniform_10_1.defaults \
        -npatterns 1100 -finput gauss.param Testing example: In the testing example, the outputs of the "evenly" trained Gaussian model will be scaled to simulate having trained on data with 10 times the number of patterns in the first class as in the second. gauss -adjust_test_priors \
        -test_priors_list 10,1 \
        -pathdata $LNKHOME/data/class \
        -finput uniform_10_1.eval \
        -fdescribe uniform_10_1.defaults \
        -npatterns 1100 -finput gauss.param In these examples, uniform_10_1 is a two dimensional data base with two univariant Gaussians with their centers separated by 2 units. The Gaussians have been unevenly sampled. There are 1000 patterns from the first and 100 patterns from the second.

Two functions occur before data is used by a classifier for training
    or testing. First data is read from a file, normalized, and stored internally. Second patterns are selected for use.

    The process of reading a pattern has three parts.  The first is the
        actual parsing of a line in the input file. In the second, the pattern is normalized using parameters read from a normalization file. Finally, if the user has requested that only certain features of the input pattern be used, those features are selected. It is important to note that if the normalization method was to apply a rotation matrix found by principal components analysis or linear discriminant analysis, features are selected from these rotated inputs, not the original raw inputs from the data file.

        The normalization parameters used during training or clustering are stored in the parameter file. These parameters are used during testing, when training a classifier that is initialized using clusters, or when training is continued for any classifier. In these cases, any normalization parameters included on the command line are ignored.

    When testing a classifier or when training some classifiers, the input
        patterns can be presented in the order they are read from the input file. Most incrementally trained classifiers, however, produce better results when the training data is presented in random order.

        When presenting data randomly, a list of all the patterns is created
            and randomized before every epoch. For any epoch, all of the data is presented in the order given in the list.

        One of the options available during training is the adjustment of
            the a priori probabilities of the classes by sampling the input data. Whenever a new pattern is to be presented to the classifier, the class of that pattern is selected first. Then, a pattern from the chosen class is selected. As happens with the random presentation, all patterns from a class are shown once before any is shown twice, although patterns from different classes can be shown with different frequencies.

        When doing cross validation, the patterns are assigned to training and
            testing sets for each fold. This assignment can be read from a file prepared by the user or the patterns can be assigned to folds automatically.

            When automatically assigning patterns to folds, lists of the patterns in each class are constructed automatically. Patterns are then assigned to testing folds. The patterns from the first class are distributed to the testing folds, cycling through the folds until there are no more patterns. Then, starting where the first class left off, the patterns in the second class are distributed to the folds. Patterns continue to be distributed until there are no more classes. This method attempts to preserve a priori class probabilities, and provides an approximately equal number of patterns per fold. If the random_cv flag is set, the patterns are distributed in a random order. Otherwise, patterns are distributed in the order they are found in the data file.

            The training set for each fold contains those patterns which are not in the test set for the fold. When doing the actual training, patterns are selected from the training set either in order, randomly, or by sampling to adjust the priors, as with normal training.

Seven main types of files are used by LNKnet classifiers

    These are the files of patterns used for training or testing the
        classifiers. Usually, for any particular data base, there are three files, a training file, an evaluation file, and a test file. The training file is usually the largest, and holds the data to be used for training classifiers. The evaluation file is used to test a classifier while tuning classifier parameters for this data base. After having used the evaluation file to tune classifier parameters, another test file is needed for reporting the error rate of this classifier on new data. This testing file is to be used infrequently, ideally only once.

        The default names of the input files are based on the name of the data base. The training file is <data_base>.train. The evaluation file is <data_base>.eval and the testing file is <data_base>.test. Thus data files in the vowel data base are vowel.train, vowel.eval, and vowel.test.

        The format of all of the data files is the same. Each file is an ASCII file of patterns with their classes, one pattern per line. Each line of the file starts with the class of the pattern. This must be an integer from 0 to noutputs-1. The remaining fields on the line are the pattern itself. There must be one floating point number for each input feature.

        An example input file (sample.train):
        0	.5	.2	1.7	17
        1	.8	1.2	1.5	5
        1	.9	1.0	1.0	2

    A  description file holds values for the data base type, the number of
        input features and input feature labels, and the number of classes and class labels for a data base. The data base type for static pattern classification data bases used by LNKnet classifiers is -class. A description file must be created for every data base. The default name for a description file is <data base>.defaults. The description file for the vowel data base is vowel.defaults.

        The description file for the example data file above (sample.defaults): describe -class -ninputs 4 -noutputs 2 \
        -labels heed,hood -input_labels A1,A2,B1,B2

    When preparing a new database for use by the LNKnet programs, a set
        of normalization parameter files should be created. They are created by the program norm. using the training input file. A normalization file is read when creating a new classifying or clustering model. The parameters from the normalization file are included in the classifier parameter file. When testing a classifier, the normalization parameters used are read from the classifier parameter file directly.

        There are four types of normalization.

        This is the default.
            The data is used as is. Normally, there is no actual normalization file for this purpose. The user should simply not set the normalize flag.

        When reading in the data, it is rescaled so that it has zero mean and
            unit variance in all ninputs dimensions. The default name for a simple normalization file is <data_base>.norm.simple. For the vowel data base, the simple normalization file is vowel.norm.simple.

        Each raw input
            pattern is multiplied by the rotation matrix stored in the normalization file. This matrix uncorrelates the features, or diagonalizes the covariance matrix of the feature set. The first dimension in the rotated space corresponds to the vector in the original space along which there is the greatest variance. The second rotated dimension corresponds to the vector, orthogonal to the first one, which accounts for the greatest amount of the remaining variance in the data. Thus, the resulting space has maximal variance in the first dimension, and decreasing variance in each of the remaining orthogonal dimensions. When performing feature selection, patterns are first rotated using matrix normalization, then the requested features are taken from the rotated patterns. The default name for a principal components normalization file is <data_base>.norm.pca.

        Each
            raw input pattern is multiplied by the rotation matrix stored in the normalization file. This matrix attempts to maximize separability of the classes in the rotated dimensions. In linear discriminant analysis, the first dimension in the rotated space corresponds to the direction in the original data along which the classes are best separated. The second rotated dimension corresponds to the next best orthogonal direction. The number of dimensions in the resulting rotated space is the minimum of the original number of dimensions and noutputs-1. As with pca normalization, features are selected from those in the rotated space. The default name for a linear discriminant normalization file is <data_base>.norm.lda.

A cross validation file contains instructions on how to divide the
    data in a particular training file for use in cross validation testing. Because all LNKnet classifiers have the ability to divide training data automatically for cross validation, a cross validation file is only required when the user desires to specify the cross validation splits by hand. The default name for a cross validation file is <input_file>.cv. Thus, if the user hand specified splits for the vowel training file, they would be stored in vowel.train.cv.

    Example: Suppose there was a training file for speech collected from three speakers. First Speaker 1 spoke and Speaker 2 replied. Then there was a pause and Speaker 2 spoke again. This time Speaker 3 replied. Speaker 3 then asked a question and Speaker 1 answered it. Finally, Speaker 2 sneezed, broke the microphone, ending the recording session. One way to use this data is to test on one speaker after training on the other two. It would be useful to eliminate the pause between the times that Speaker 2 spoke and the noise at the end. It might also be convenient to add a pause between Speaker 3's first statement and her question. The cross validation file below would accomplish this. cross_valid -nfolds 3 -nsplits 9 \
      -cv_splits 0:14,15:29,30:36,37:49,50:70,-1:-1,71:84,85:90 \
      -cv_train_mask 010110100,100010110,110100010 \
      -cv_test_mask 100000010,010100000,000010100 The splits divide the data into these nine parts: Speaker 1's question, which is patterns 0 through 14, Speaker 2's answer, which is 15 patterns starting at 15, the pause, Speaker 2's question, Speaker 3's answer, the fake pause, which is specified by the "empty split", -1:-1, Speaker 3's question, Speaker 1's answer, and the noise at the end. The first cross validation fold is trained on the speech from Speakers 2 and 3 and is tested on the data from Speaker 1. Likewise, fold 2 is tested on Speaker 2 and trained on 1 and 3 and fold 3 is tested on 3 and trained on 1 and 2. Because the pauses and noise are never trained on or tested on, they could have been eliminated.

The flags available in a cross valid file are:

    -nfolds n
        The number of classification models or cluster sets to train and test.

    -nsplits n
        The number of contiguous sections the data is split into. These sections are specified in cv_splits and assigned to training and test folds in cv_train_mask and cv_test_mask.

    -cv_splits n[,n]...
        cv_splits is an integer list of the line numbers in finput which begin and end each section, or split, of the data to be used in the cross validation folds. The list is made up of pairs of line numbers, counting the lines starting at 0. To make the list easier for users to read, either colon or comma, or some combination of both can be used as the list delimiter. There should be 2*nsplits entries in the list. To specify a single pattern, give its number as both the start and end of a split. To make a split with no patterns in it, which can be useful when creating the train and test masks, add the pair -1:-1.

    -cv_train_mask <list of bit vectors>
        A list of nfolds strings. Each string has nsplits characters, all 1's and 0's. Each string specifies the sections described in cv_splits to be included in each cross validation training set. For each section, a 1 includes it in the training set for the given fold, a 0 excludes it. There are no spaces in the argument to cv_train_mask. The bit vectors are separated by commas.

    -cv_test_mask <list of bit vectors>
        A list of nfolds strings, each nsplits long, as described for cv_train_mask. This mask specifies the sections described in cv_splits to be included in each cross validation test set.

All the information needed to reconstruct a classifier for testing or
    for continuing training is stored in a parameter file. Parameter files are ASCII and can be examined using an editor. Parameter files are written using a subroutine <classifier>_write_model(). There is a routine <classifier>_read_model() used for reading the parameters and reconstructing the classification model from them. These routines use the subroutines param_putblk() and param_getblk() from the nnlib library. To write your own subroutine to read in the parameter file, look at how param_putblk() and param_getblk() were used in in the source code for the particular classifier and look at the documentation for param_getblk() in the nnlib library src directory.

    The parameter files for clustering programs all have the same format, to allow them all to be used by the classifiers that need them.

During testing and during incremental training, pattern by pattern
    classification results are written to an error file. The contents of this error file are controlled by the value of verror. The following settings for verror are recognized by the classifiers.

    0 - Do not produce an error file.

    1 - Produce a "results" error file. There is a line in the error file for each pattern tested by the classifier. There are 5 fields in each line. The fields are:
        input pattern number, class of the pattern, class selected by the classifier, whether the class selected was wrong, and the cost of the output vector. This cost is usually the squared error of the classifier outputs against a desired output of 1 for the correct class and 0 for all others. MLP_lnk(1) and IRBF_lnk(1) can also use other cost functions.

    2 - Produce a "results+outputs" error file. There is a line in the error file for each pattern tested by the classifier. There are 5 + noutputs fields in each line. The first five fields are as in the "results" file. The next noutputs fields are the outputs produced by the classifier for this pattern. These outputs are either Bayesian probabilities or likelihoods, depending on the classifier. If the log_outputs flag is set, the outputs printed will be base 10 logarithms of the probabilities. 3 - Produce a "results+outputs+inputs" error file. There is a line in the error file for each pattern tested by the classifier. There are 5 + noutputs + ninputs fields in each line. The first 5 + noutputs fields are as in the "results+outputs" file. The last ninputs fields are the inputs given to the classifier for this pattern.

        Example: This is an example of a results+outputs+inputs error file. The first line of this example is a set of labels for the fields in the file and would not be included in a real error file. There are two inputs and two outputs and three patterns were tested
        Pat#	class	guess	wrong?	cost	out0	out1	in0	in1
        0	0	1	1	0.565	0.2 	0.7	1.1	0.5
        1	1	1	0	0.0058	0.1	0.96	0.6	0.8
        2	0	0	0	0.1	0.8	0.4	1.5	0.2

LNKnet programs print information about a classification run to
    the standard output and to the standard error output. When running LNKnet programs from the LNKnet graphical interface, this information is also written to a log file. The quantity of information printed depends on the setting of verbose.

    A line to mark the start of the program.

    The command used to call this program.  This is an expanded version of
        the command, with the default values of those flags which were not set shown. The start time for execution is also shown.

    If this is an incremental training run, some indication of the
        classifier's status for each epoch.

    A confusion matrix showing results from any testing done during this
        run.

    An error summary by class for any testing done during this run.

    An overall error rate for testing during this run.

    A line to mark the end of the program.

OPTIONS
The following options are recognized with spaces separating an option and any following items:

--  A double dash on the command line prints a list of all classifier flags and their arguments without running the program.

-train
    Perform training. If -create is set, the classifier is trained from scratch. Otherwise, an existing parameter file is read and training is continued on that classifier.

-create
    Creates a new error file. Otherwise, pattern by pattern errors are appended to an existing error file. If the -train flag is on, a new classifier model is created.

-pathexp directory
    The base directory used for writing the files ferror and fparam A slash "/" is automatically added to the end of the base directory if it doesn't end with a slash. Example: -pathexp ./

-ferror file
    Name of the file in pathexp for storing classification results in testing or training. Example: -ferror X1mlp.err.train

-fparam file
    Name of the file in pathexp for storing classifier parameters. Example: -fparam X1mlp.param

-pathdata directory
    The base directory used for finding the data files finput, fdescribe, fnorm, and fcross_valid. A slash "/" is automatically added to the end of the base directory if it doesn't end with a slash. Example: -pathdata lnknet/data/class

-finput file
    This flag is required. The ASCII file in pathdata with the data to be used for either training or testing a classifier or clusterer. The format and naming conventions for input files are described above. Example: -finput vowel.train

-fdescribe file
    This file is required. The file in pathdata which contains the values for the data base type, number of input features, input feature labels, number of output classes, and the class labels. Description files are described in more detail above. Example: -fdescribe vowel.defaults

-npatterns n
    n is the number of patterns to read from finput. If this number is less than the number of lines in finput, only the first npatterns patterns will be used by the classifier.

-ninputs n
    n is the number of dimensions of each normalized pattern to be used by the classifier. If this flag is not set, all available data dimensions will be used. If this flag is set to a number less than the number of inputs given in the data base description file, only the first ninputs dimensions will be used. If a list of dimensions to be used is specified by features, ninputs should be the length of the features list. If testing a classifier model or continuing the training of an existing model, this flag has no effect.

-features n[n,]...
    An integer list of the dimensions of the normalized data to be used by the classifier in training. These dimensions are counted starting at 0, meaning that in the case of a three dimensional data base, the entries in the features list can be 0, 1, or 2. If testing a classifier model or continuing the training of an existing model, this flag has no effect.
    Example: -features 3,0,2
    The fourth, first, and third dimensions of the data will be used in this classifier in the order given.

-normalize
    When reading in the data, normalize it using the parameters present in fnorm. If testing a classifier model or continuing the training of an existing model, this flag has no effect.

-fnorm file
    The file in pathdata which has the normalization parameters to be used when normalizing the data in the input file. Normalization files are described above. Example -fnorm vowel.norm.simple

-cross_valid n
    If the train flag is not set, do n fold cross validation on the data. This means that the data will be split into n parts and n classification models or sets of clusters will be trained and tested on the data in finput. The patterns are automatically assigned to folds as described above. If the flag cross_valid_from_file is set, this flag is ignored.

-random_cv
    If this is an n fold cross validation experiment and patterns will be automatically assigned to folds, randomize the order in which patterns from each class are assigned to folds. This randomizes the pattern assignments but does not change the class probabilities of the folds.

-cross_valid_from_file
    If the train flag is not set, do cross validation on the data using the flags in cross_valid_file to determine which patterns in finput are in each training and testing fold.

-fcross_valid file
    The file in pathdata which contains flags which specify the splits and folds used in cross validation when cross_valid_from_file is set. An example of a cross validation file is given above, in the description section of this manual page. Example: -fcross_valid vowel.train.cv

-random
    Randomize the order of the data before each epoch of training. This flag has no effect when testing.

-seed n
    Initialize the random number generator using n as the seed.

-adjust_train_priors
    Sample the training data to provide different a priori probabilities for the classes. All of the data is read in and the number of patterns from each class is found. When training, each class is selected with either uniform probability or with the probability given in train_priors_list. A pattern from the selected class is then chosen and used to train the classifier.

-priors_npatterns n
    When doing training and adjust_train_priors is set, show n patterns to the classifier during each epoch. These patterns will be selected from the npatterns patterns read from finput to give uniform probability to each class or with the class probabilities given in train_priors_list.

-train_uniform_priors
    When adjust_train_priors is set, let each class have equal probability. When this flag is set, train_priors_list is ignored.

-train_priors_list x[,x]...
    If train_uniform_priors is not set, this is the list of class probabilities to be used while adjusting a priori class probabilities during either training or testing. This must be a comma delimited list of noutputs floating point numbers. The "probabilities" are normalized to sum to one before being used by the classifier, so they can be any positive floating point number.
    Example:
    -train_priors_list 5,15,20,60
    which is equivalent to
    -train_priors_list 0.05,0.15,0.2,0.6

-adjust_test_priors
    During testing, scale continuous classifier outputs to simulate having trained on data with uniform a priori class probabilities or using the probabilities given in test_priors_list. This flag does not apply when doing training or cross validation.

-test_uniform_priors
    When adjust_test_priors is set, let each class have equal probability. When this flag is set, test_priors_list is ignored.

-test_priors_list x[,x]...
    If test_uniform_priors is not set, this is the list of class probabilities to be used while adjusting a priori class probabilities during either training or testing. This must be a comma delimited list of noutputs floating point numbers. The "probabilities" are normalized to sum to one before being used by the classifier, so they can be any positive floating point number.
    Example:
    -test_priors_list 5.0e-1,1.5,2.0,6.0
    which is equivalent to
    -test_priors_list 0.05,0.15,0.2,0.6

-debug n
    The value of n determines the quantity of debugging messages printed out during runs of a classifier.

-verbose n
    The value of n determines the quantity of classifier status messages and error summary information printed out during runs of a classifier. What printed at which verbosity levels is described above.

-verror n
    The value of n determines the amount of data put in the error file ferror. The possible values for n are described above.

-log_outputs
    If verror is 2, print the log base 10 values of the output probabilities to the error file. Otherwise, these outputs are linear probabilities or likelihoods.

SEE ALSO
Manual pages for individual classifiers and clusterers:
bintree_lnk(1), cknn_lnk(1), gauss_lnk(1), gmix_lnk(1), histogram_lnk(1), hyper_lnk(1), irbf_lnk(1), knn_lnk(1), lvq_lnk(1), mlp_lnk(1), nbayes_lnk(1), nc_class_lnk(1), parzen_lnk(1), rbf_lnk(1), svm_lnk(1), em_clus_lnk(1), kmeans_lnk(1), ran_clus_lnk(1), lead_clus_lnk(1)

HISTORY
First written 13 May 1992 by Linda Kukolich
Modified 2 July 1993 by Linda Kukolich - final editing
Modified 19 November 1993 - added bintree, changed defaults file descriptions
Modified 21 June 1994 - added pathdata to examples, one description file per data base

BUGS
Send bug reports to Linda Kukolich (kukolich@sst.ll.mit.edu)