FEAT_SEL(1)

NAME
feat_sel - Feature Selection Program

SYNOPSIS
Selects the best features from the given data set by building and testing a series of K Nearest Neighbor classifiers. Each classifier is tested using leave-one-out cross-validation.

feat_sel [ -pathexp directory ] [ -search_fparam file ]
[ -pathdata directory ] [ -finput file ]
[ -fdescribe file ] [ -npatterns n ]
[ -normalize ] [ -fnorm file ] [ -debug n ]
[ -search_verbosity n ] [ -search_type n ] [ -k n ]
[ -stop_early ] [ -search_n n ]

EXAMPLE
feat_sel -finput digit1.train \
-fdescribe digit1.train.defaults -npatterns 70 -ninputs 22 \
-noutputs 7

DESCRIPTION
Stores a list of features of a data base in order of their effectiveness in classification. The effectiveness of each tested feature list is found using a K nearest neighbor classifier and leave-one-out cross validation testing. feat_sel has three ways of conducting the search for the best features, a forward search, a backward search, and a forward-and-back search.

In the forward search, a one feature classifier is built and tested for each feature. The best solo feature is selected as the first feature. Then, each remaining feature is used with the first feature to build a two feature classifier. The best of these is selected as the second feature and the search continues until all features have been selected.

In the backward search, the program starts with all n features selected. n classifiers are then built and tested, each with one feature left out. The feature left out of the best n-1 feature classifier is made the nth feature. To select the (n-1)th feature, the n-1 n-2 feature classifiers are tested. The search continues until only one feature remains, which is chosen as the first feature.

The forward-and-back search combines the other two search methods. First, two features are selected, one at a time, as in the forward search. Next, one feature is left out, as with the backward search. Then two more are added and one left out until all of the features have been added.

OPTIONS
-- Prints a list of all flags and arguments.

-pathexp directory
    The base directory used for writing the parameter file, search_fparam. A slash "/" is automatically added to the end of the base directory if it doesn't end with a slash.

-search_fparam file
    Name of a file in pathexp for storing feature search results. This file is written using the routine feat_sel_write_model() found in feat_sel_pack.c. It can be read using feat_sel_read_model() or by using the parameter file handling routines in the library nnlib.a.

-pathdata directory
    The base directory used for finding the data files finput, fdescribe, and fnorm. A slash "/" is automatically added to the end of the base directory if it doesn't end with a slash.

-finput file
    The ASCII training file in pathdata with the data to be used for the feature search. The format and naming conventions for input files are described on the classifier(1) manual page.

-fdescribe file
    The ASCII file in pathdata which contains the number of input features and the number of output classes for the data base. Description files are described on the classifier(1) manual page.

-npatterns n
    n is the number of patterns to read from finput. If this number is less than the number of lines in finput, only the first npatterns patterns will be used by the classifier.

-normalize
    When reading in the data, normalize it using the parameters present in fnorm. The features are selected from those in the normalized space.

-fnorm file
    The file in pathdata which has the normalization parameters to be used when normalizing the data in the input file. Normalization files are described in more detail on the classifier(1) manual page.

-debug n
    The value of n determines the quantity of classifier status messages printed out during the feature search.

-search_verbosity n
    The value of n determines the quantity of search status messages printed during the feature search.

    0   Print out the feature list that got the best error rate and the list of all sorted features.

    1   Also print out the feature list found for each number of input features.

    2   Also print out the the results for each feature list tested.

-search_type n
    n determines the type of search done.

    0 Forward Search
        Build the feature list by selecting one feature to add each time.

    1 Backward Search
        Starting with all features, select features to remove one at a time.

    2 Forward and Back
        Combine a forward and a backward search. Add two features, one at a time, then take away one. Continue this way until all of the features have been added.

-k n
    The classifier used by the feature search algorithm is a K Nearest Neighbor classifier, tested using leave one out cross validation. n is the k to use in the classifier.

-stop_early
    Stop the feature search when search_n features has been reached. If searching forward, or using the forward-and-back search, stop when search_n features have been selected. If searching backward, search until only search_n features remain. In the parameter file, the whole list of features will be present. If this was a forward search, the features that remained after the first search_n were selected will be listed in random order. If this was a backward search, the features that remained after search_n features were removed will be listed in random order. In both cases, these "un-selected" features will be assigned an error rate of 100% in the parameter file.

-search_n n
    The number of features found when stopping the search early. This flag only has an effect when stop_early is selected.

SEE ALSO
classifier(1)

HISTORY
First written 24 June 1992 by Linda Kukolich

BUGS
Send bug reports to Linda Kukolich (kukolich@sst.ll.mit.edu)