UMass Machine Learning and Friends Lunch | Main / Selection Of Consistent Roots To The Likelihood Equation In Finite Mixtures Of Location-scale Distributions

Finite mixtures of location-scale distributions such as normal mixture distributions are attractive in capturing any nonnormal features in the data caused by underlying group structure. However, it is well known that the maximum likelihood estimation in the location-scale mixture distributions is problematic due to the unbounded likelihood (singularities) and the existence of multiple roots to the likelihood equation.

There are standard methods dealing with the unbounded likelihood in order to still obtain a (restricted/penalized) MLE. They are based either on restricting the parameter space or on penalizing the likelihood function. Although these approaches seems reasonable as well, they suffer from some important drawbacks. Despite the non-existence of the global MLE, the likelihood equation contains a consistent root that has the same properties as an MLE. However, in practice, the largest local likelihood criterion is not reliable due to the existence of spurious maximum, especially when the sample size is not large relative to separation of the components.

In this talk we propose a method to avoid such singularities in the mixture likelihood and to choose a reasonable root to the likelihood equation. Our proposed method, score-based k-deleted likelihood, will be employed as a criterion for selecting a local maximum among all found roots to the likelihood equation. Numerical examples are provided in the finite-sample cases, showing the performances of score-based k-deleted likelihood compared to a standard approach.