PLOT_DETECT(1)

NAME
plot_detect - Plot detection rate versus false alarm curve (ROC) for a LNKnet classifier.

SYNOPSIS
Plot a detection rate versus false alarm rate curve for spotting problems where classifiers either ignore a pattern or detect a pattern as being a member of a target class.

plot_detect [-target n] [-pathexp directory] [-ferror file]
[-fplot file] -noutputs n [-reject x] [-xmin x]
[-xmax x] [-xstep x] [-ymin x] [-ymax x] [-ystep x]
[-title string] [-verbose n] [-table_begin x]
[-table_end x] [-table_step x] [-mac] [-fmaclines file]
[-no_graphics] [-no_borders]

DESCRIPTION
In spotting or detection problems, a score is calculated for a target class for every input pattern presented to a classifier. This score is compared to a threshold, and the pattern is detected and assigned to the target class if the score exceeds the threshold. This type of detection or spotting process generates a correct detection or hit if the pattern is a member of the target class and the score is above threshold. A false alarm is generated if the pattern is not a member of the target class and the score is above the threshold. A miss occurs if the pattern is from the target class and the score is below threshold and a correct rejection occurs if the pattern is not from the target class and the score is below threshold. This program analyzes the performance of classifiers on detection problems by producing a plot and table containing the percent target patterns correctly detected as a function of the percentage of non-target patterns which generate false alarms.

Plots are generated using data from a LNKnet error file. The error file verbosity flag verror, must be greater than or equal to 2 during testing to generate an error versus rejection rate plot. Plots can only be generated for classifiers with continuous outputs and sufficient storage space must be available to store the error file until plot_detect can be run.

EXAMPLE
This example tests a Multi-Layer Perceptron classifier using test data and creates a detection versus false alarm plot.

mlp -train -create -pathdata $LNKHOME/data/class \
-finput vowel.train -fdescribe vowel.defaults \
-npatterns 338 -fparam mlp.param -nodes 2,25,10 -epochs 100

mlp -pathdata $LNKHOME/data/class -finput vowel.test \
-fdescribe vowel.defaults -npatterns 167 \
-ferror mlp.err.test -verror 2 -fparam mlp.param

plot_detect -target 0 -pathexp ./ -ferror mlp.err.test \
-fplot reject.plot -noutputs 10 -verbose 2

OPTIONS

--  Prints a list of all flags and arguments.

-target n
    Class number which should be considered to be the target. Class numbers range from 0 to the number of classes minus 1. The default target is class 0.

-pathexp directory
    Directory where ferror and fplot are stored.

-ferror file
    File where the trial-by-trial classification test results are stored. The error file verbosity flag verror, must be greater than or equal to 2 during testing to generate this error file.

-fplot file
    File for storing plot.

-noutputs n
    There are outputs for n classes stored in the error file.

-reject x
    The fraction of patterns to reject before generating the ROC curve. That is, if the outputs for N patterns are stored in the error file and if -reject is set to 0.25, the N/4 patterns with the lowest scores will not be included in the plot.

-xmin x
    The lower limit of the percent false alarm axis for the plot. The default value is 0.

-xmax x
    The upper limit of the percent false alarm axis for the plot. The default value is 100.

-xstep x
    The space between tic marks on percent false alarm axis for the plot. The default value is 10.

-ymin x
    The lower limit of the percent detection axis for the plot. The default value is 0.

-ymax x
    The upper limit of the percent detection axis for the plot. The default value is 100.

-ystep x
    The space between tic marks on the percent detection axis for the plot. The default value is 10.

-title string
    A plot label to display at the top of the plot.

-verbose n
    Controls amount of printout.

    0 - Print nothing to standard output

    1 - Print area under ROC curve to standard output

    2 - Print a sampled table to standard output using the table_begin, table_end, and table_step arguments to determine the contents of the table.

    3 - Print a table of all the points on the ROC curve

-table_begin x
    When the verbosity is set to 2, start the table at a false alarm rate of x.

-table_end x
    When the verbosity is set to 2, the false alarm rate of the final entry in the table is less than or equal to x.

-table_step x
    When the verbosity is set to 2, increase the false alarm rate by x for each entry of the table.

-mac
    If this flag is set, the points in the plot will be written to the file -fmaclines.

-fmaclines file
    This is an ASCII file containing the data used to create the detection versus false alarm rate plot. This file is equivalent to the the table printed for a verbosity of 3. This file can be used to generate plots on a MacIntosh or other PC using a spread-sheet graphing program such as Excel or DeltaGraph.

-no_graphics
    Generate the plot file but do not display it in an olxplot window.

-no_borders
    Only draw the ROC curve. Do not draw the axes, the labels, or the titles.

SEE ALSO
classifier(1), plot_reject(1), gauss_lnk(1), gmix_lnk(1), irbf_lnk(1), rbf_lnk(1), mlp_lnk(1), histogram_lnk(1), parzen_lnk(1)

HISTORY
First written July 1993.
Modified 19 April 1995, added table_ flags.

BUGS
Send bug reports to Linda Kukolich (kukolich@sst.ll.mit.edu)