EM_CLUS_LNK(1)

NAME
em_clus_lnk - Binary splitting Clusterer using the Estimate Maximize algorithm.

SYNOPSIS
A program for finding K cluster centers for the input data using the EM algorithm as in the Gaussian mixture classifier. The EM clusterer maximizes the likelihood of the data, estimates the means and variances of a Gaussian mixture distribution, and ignores mixture probabilities.

em_clus_lnk [ -cluster_by_class ] [ -ncenters n ]
[ -ncenters_list n[,n]... ] [ -split_percentage x ]
[ -add_random_offset ] [ -max_iteration n ]
[ -stop_percentage x ] [ -reduce_step n ]
[ -var_spread x ] [ -minvar x ] [ -use_kmeans ]
[ -kmeans_stop_percentage x ] [ -classifier_options ]

EXAMPLE
em_clus_lnk -ncenters 16 -train -pathdata $LNKHOME/data/class \
 -finput vowel.train -fdescribe vowel.defaults \
-npatterns 338

DESCRIPTION
em_clus_lnk training starts with a single cluster at the mean of the data. This cluster is split in two and the means and variance of the new clusters are iteratively trained. These two clusters are then split and the process continues until the requested number of clusters have been trained. If the number of clusters requested is not a power of two, the nearest power of two above the request is used. Then, clusters are taken away and the remaining clusters are iteratively trained until the requested number has been reached.

When splitting a cluster, the initial means of new clusters are shifted a given percentage of the variance of that cluster away from the mean of the original cluster. They use the variance of the old cluster as their initial variance.

When iteratively training the means and variances of a set of clusters, an average log likelihood is calculated which is the average of the log of the products of the Gaussian outputs of all the clusters for that point. If the average log likelihood changes by less than a certain percentage each iteration, the algorithm stops. Likewise, if more than a certain number of iterations have gone by with this set of clusters, the algorithm stops. At this point, the existing clusters are split, pruned, or the program is finished.

During the iterative training of the clusters each pattern shifts the means and variances of each cluster in proportion to its Mahalanobis distance from that cluster. The Mahalanobis distance of a point from a cluster is the Gaussian output of that cluster for that point. When clustering by class, the input patterns only affect the clusters in their own class.

OPTIONS
-- Prints a list of all flags and arguments.

-cluster_by_class
    Train a separate set of clusters for each class.

-ncenters n
    n is the total number of clusters to find or the number of clusters to find for each class.

-ncenters_list n[,n]...
    The number of clusters to find for each of the noutputs classes.
    Example: -ncenters_list 3,6,2,1,5

-split_percentage x
    When splitting a cluster x is the percentage of its variance to perturb the two clusters created from it.

-add_random_offset
    When spitting a cluster, perturb the means plus or minus an extra 5% of the split percentage when splitting. Breaks up symmetries in certain data bases.

-max_iteration n
    Do at most n iterations after a split or after reducing the number of clusters.

-stop_percentage x
    When the average log likelihood of the data changes by less than stop_percentage from one iteration to the next, split all of the clusters, reduce the number of clusters, or stop the algorithm, whichever is appropriate.

-reduce_step n
    When finding a non-binary number of clusters, prune n clusters at a time before running the EM algorithm again.

-var_spread x
    x is the amount to scale the variance of each cluster just after a split.

-minvar x
    x is the minimum value of the variance in any dimension in any cluster.

-use_kmeans
    Initialize em_clus_lnk clusters by using the kmeans algorithm. When kmeans has found the requested number of clusters, run the em_clus_lnk algorithm on them until it converges.

-kmeans_stop_percentage x
    When using kmeans for initialization, when the average squared error changes by less than x, split, prune, or stop, whichever is appropriate.

-classifier_options
    em_clus_lnk accepts the general classifier arguments described on the classifier(1) manual page. Certain arguments are not used by the program, and so are not available.

    Unavailable arguments from general list:

    -create
        Because training on a model cannot be continued and there is no testing mode, the create flag is always set.

    -adjust_test_priors -test_uniform_priors -test_priors_list
        There is no testing mode.

    -log_outputs, -verror, -ferror
        No error file is created.

    -random
        This flag only has an effect when adjust_train_priors is set.

REFERENCE
Kenney Ng, A Comparative Study of the Practical Characteristics of
 Neural Network and Conventional Pattern Classifiers, Bachelor's and Master's Thesis, Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Cambridge, MA, May 1990. Pages 43-48.

SEE ALSO
classifier(1), em_clus_plot_bound(1)

HISTORY
First written 26 May 1992 by Linda Kukolich

BUGS
Send bug reports to Linda Kukolich (kukolich@sst.ll.mit.edu)