2006 Digital Symposium Collection

Maximum weighted likelihood via rival penalized EM for density mixture clustering with automatic model selection

Yiu-ming Cheung
View Paper (PDF)

Return to June 2005, Volume 17, Issue 6

Abstract

Expectation-maximization (EM) algorithm (A.P. Dempster et al., 1977) has been extensively used in density mixture clustering problems, but it is unable to-perform model selection automatically. This paper, therefore, proposes to learn the model parameters via maximizing a weighted likelihood. Under a specific weight design, we give out a rival penalized expectation-maximization (RPEM) algorithm, which makes the components in a density mixture compete each other at each time step. Not only are the associated parameters of the winner updated to adapt to an input, but also all rivals' parameters are penalized with the strength proportional to the corresponding posterior density probabilities. Compared to the EM algorithm (A.P. Dempster et al., 1977), the RPEM is able to fade out the redundant densities from a density mixture during the learning process. Hence, it can automatically select an appropriate number of densities in density mixture clustering. We experimentally demonstrate its outstanding performance on Gaussian mixtures and color image segmentation problem. Moreover, a simplified version of RPEM generalizes our recently proposed RPCCL algorithm (Y.M. Cheung, 2002) so that it is applicable to elliptical clusters as well with any input proportion. Compared to the existing heuristic RPCL (L. Xu et al., 1993) and its variants, this generalized RPCCL (G-RPCCL) circumvents the difficult preselection of the so-called delearning rate. Additionally, a special setting of the G-RPCCL not only degenerates to RPCL and its Type A variant, but also gives a guidance to choose an appropriate delearning rate for them. Subsequently, we propose a stochastic version of RPCL and its type A variant, respectively, in which the difficult selection problem of delearning rate has been novelly circumvented. The experiments show the promising results of this stochastic implementation.