Model Identifiability and Boundary Solutions Sample Clauses

Model Identifiability and Boundary Solutions. When estimating ψ for a mixture distribution, model identifiability needs to be con- sidered. In general, a parametric family of densities f (yi; ψ) is identifiable if distinct values of the parameter ψ determine distinct members of the family of densities {f (yi; ψ) : ψ ∈ Ω}, where Ω is the specified parameter space[54]. Based on this definition of identifiability, a mixture distribution would not be identifiable because f (yi; ψ) is invariant under the g! permutations of the component labels in ψ. Thus, the definition of identifiability is slightly modified in the context of mixture distri- butions. Specifically, finite mixture models are said to be identifiable for ψ ∈ Ω if f (yi; ψ) = f (yi; ψ∗) if and only if ψ = ψ∗ up to a permutation of the component labels. In practice, a constraint is sometimes imposed on ψ that uniquely determines the component labels after estimation. In the context of finite mixture models, nonidentifiability due to overfitting also needs to be considered [33]. To illustrate nonidentifiability due to overfitting, consider a finite mixture model where the true number of components is C = 2. This mixture model can also be written with C = 3 components if the third component has a weight of zero, i.e. f (yi; ψ) = π1f1(yi; θ1) + π2f2(yi; θ2) + 0 × f3(yi; θ3), or if two of the components are the same, i.e. f (yi; ψ) = π1f1(yi; θ1) + (π2 − π3) f2(yi; θ2) + π3f3(yi; θ2). More generally, Xxxxxxxx(1994) [8] notes that any mixture with C −1 components de- fines a nonidentifiable subset in the larger parameter space corresponding to mixtures with C components. Although nonidentifiability due to label switching can be easily addressed, nonidentifiability due to overfitting can be more problematic. Specifically, overfitting can result in numerical difficulties because the matrix of second derivatives will be close to singular. As such, Xxxxxxxx suggests that, as a practical matter, it is usually preferable to reduce the number of components in the mixture rather than to work in the full-dimensional space.