Bayesian Estimation of Finite Mixture Models Sample Clauses

Bayesian Estimation of Finite Mixture Models. An alternate way to conceptualize and estimate a finite mixture model is to use a Bayesian approach. Assume that there exists a prior distribution for all unknown parameters in the mixture model, p (ψ). Then, define the posterior density as p (ψ|y) ∝ p (y|ψ) p (ψ) . Although the Bayesian approach tends to reduce the risk of obtaining spurious modes in cases where the EM algorithm leads to degenerate solutions[33], there are no natural conjugate priors available for the mixture likelihood function. As such, the posterior density p (ψ|y) does not belong to any standard distributional family. Thus, until the development of Markov Chain Monte Carlo (MCMC) methods, Bayesian estimation approaches for finite mixture models were infeasible. As computational resources have increased, MCMC methods for finite mixture models have become more common; however, the Bayesian approach does present some unique challenges. For example, as described above, the likelihood function associated with a finite mixture model is invariant under a permutation of the com- ponent labels. Although maximum likelihood estimation via the EM algorithm is not affected by potential switching of component labels during different iterations, label switching can be problematic for Bayesian estimation, which relies on the simulation of realizations of ψ from posterior distributions. In addition, priors on the mixing proportions have been used to draw the mixing proportions away from the boundary of the parameter space and to avoid the numerical issues that sometimes arise due to boundary solutions. This should be done with care, however, since it can eliminate the possibility of reducing the number of components when the model is actually overfit and informative priors tend to force too many distinct components[33]. A more comprehensive overview of the Bayesian approach to finite mixture models, the challenges it presents, and methods for overcoming some of these challenges can be found in Xxxxxxxxx-Xxxxxxxxx(2006)[33]. It should also be noted, that recently Bayesian approaches to finite mixture models for repeated measurements have begun to appear in the literature (see, for example, [22]). These approaches are often referred to as Bayesian growth mixture models. In growth mixture model approaches, the latent class variable is not directly identified by the feature variables. Instead, the latent class variable captures heterogeneity in the growth model parameters. In this context, it is important to note th...