Grant: Discovery Grants Program - Individual
University of British Columbia
$11,000.00
Likelihood-based tests for the Number of Components in Finite Mixture Models
EF 2016-17
Résumé
Finite mixtures of normal distributions have been used in numerous empirical applications across various fields such as biological, physical, and social sciences, including finance, economics, and marketing. Mixture-of-expert models with normal component distribution, which have been used in numerous regression, classi?cation, and fusion applications in healthcare, ?nance, surveillance, and recognition, can be viewed as finite mixture of normal regression models. The number of components is an important parameter in applications of finite mixture normal regression models. In economics applications, the number of components often represents the number of unobservable types or abilities. In many other applications, the number of components signifies the number of clusters or latent classes in the data.
Despite its importance, testing for the number of components in finite mixture normal regression models has been a long-standing unsolved problem because the standard asymptotic analysis of the likelihood ratio test (LRT) statistic breaks down due to problems such as non-identifiable parameters and the true parameter on the boundary of the parameter space. In normal mixtures with unequal variances, the asymptotic distribution of the LRT statistic remains an open question because normal mixtures have an additional undesirable mathematical property that invalidates key assumptions in the existing works, such as ``the lack of strong identifiability'' as discussed by Chen (1995, Annals of Statistics) and the infinite Fisher information with respect to mixing proportion.
This project studies likelihood-based testing of the null hypothesis of m components against the alternative of (m+1) components in general finite normal mixture models with a vector mixing parameter and a structural parameter, including finite mixture normal regression models with heteroskedastic components. Our project intend to make the following contributions.
We develop an orthogonal parameterization that extracts the direction in which the Fisher information matrix is singular. Under this reparameterization, the log-likelihood function is locally approximated by a quadratic form of polynomials of the reparameterized parameters, leading to a simple characterization of the asymptotic distribution of the LRT statistic. Based on this reparameterization, we derive the asymptotic distribution of the LRT statistic for testing the null hypothesis of m components for m larger than 2 in a mixture model with a multidimensional mixing parameter and a structural parameter.
Implementing the LRT has, however, practical difficulties because (i) in some mixture models that are popular in applications (e.g., Weibull duration models and normal mixture models), the Fisher information may not be finite, (ii) the asymptotic distribution depends on the choice of the support of the parameter space, and (iii) simulating the supremum of a Gaussian process is computationally challenging because of the curse of dimensionality.
To circumvent these difficulties, We propose a modified EM test by building on this local quadratic representation and extending the EM approach pioneered by Li and Chen (2010, Journal of the American Statistical Association). Given our preliminary results, we expect that the asymptotic null distribution of the proposed modified EM test statistic will be easily simulated. Furthermore, the modified EM test does not suffer from the infinite Fisher information problem.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2014-06221
Name nom
Kasahara, Hiroyuki
Department département
Economics
Organization id
2
Institution établissement
University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2016
Competition year année de concours
2014
Award amount
11000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1212
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Social sciences
Area of application fr
Sciences sociales
Research subject code
3001
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistical theory
Research subject fr
Théorie de statistique
Partie
3
Nb partie
5
Application title
Likelihood-based tests for the Number of Components in Finite Mixture Models
Keyword
NSERC
Application summary
Finite mixtures of normal distributions have been used in numerous empirical applications across various fields such as biological, physical, and social sciences, including finance, economics, and marketing. Mixture-of-expert models with normal component distribution, which have been used in numerous regression, classi?cation, and fusion applications in healthcare, ?nance, surveillance, and recognition, can be viewed as finite mixture of normal regression models. The number of components is an important parameter in applications of finite mixture normal regression models. In economics applications, the number of components often represents the number of unobservable types or abilities. In many other applications, the number of components signifies the number of clusters or latent classes in the data.
Despite its importance, testing for the number of components in finite mixture normal regression models has been a long-standing unsolved problem because the standard asymptotic analysis of the likelihood ratio test (LRT) statistic breaks down due to problems such as non-identifiable parameters and the true parameter on the boundary of the parameter space. In normal mixtures with unequal variances, the asymptotic distribution of the LRT statistic remains an open question because normal mixtures have an additional undesirable mathematical property that invalidates key assumptions in the existing works, such as ``the lack of strong identifiability'' as discussed by Chen (1995, Annals of Statistics) and the infinite Fisher information with respect to mixing proportion.
This project studies likelihood-based testing of the null hypothesis of m components against the alternative of (m+1) components in general finite normal mixture models with a vector mixing parameter and a structural parameter, including finite mixture normal regression models with heteroskedastic components. Our project intend to make the following contributions.
We develop an orthogonal parameterization that extracts the direction in which the Fisher information matrix is singular. Under this reparameterization, the log-likelihood function is locally approximated by a quadratic form of polynomials of the reparameterized parameters, leading to a simple characterization of the asymptotic distribution of the LRT statistic. Based on this reparameterization, we derive the asymptotic distribution of the LRT statistic for testing the null hypothesis of m components for m larger than 2 in a mixture model with a multidimensional mixing parameter and a structural parameter.
Implementing the LRT has, however, practical difficulties because (i) in some mixture models that are popular in applications (e.g., Weibull duration models and normal mixture models), the Fisher information may not be finite, (ii) the asymptotic distribution depends on the choice of the support of the parameter space, and (iii) simulating the supremum of a Gaussian process is computationally challenging because of the curse of dimensionality.
To circumvent these difficulties, We propose a modified EM test by building on this local quadratic representation and extending the EM approach pioneered by Li and Chen (2010, Journal of the American Statistical Association). Given our preliminary results, we expect that the asymptotic null distribution of the proposed modified EM test statistic will be easily simulated. Furthermore, the modified EM test does not suffer from the infinite Fisher information problem.