Variable Selection in Factor Analysis.

Factor analysis is commonly used to describe the covariance structure for a group of variables through a set of underlying latent factors. Variables that are associated with the same factor are correlated with one another. It is often desirable to remove variables from the model that do not contribute to any factors, which can be accomplished by identifying and eliminating variables that are uncorrelated with any others. Traditionally, variable selection in factor analysis is done by removing variables based on the magnitude of their factor loadings after the model has been estimated. However, this approach relies on somewhat arbitrary cutoffs, with ambiguous cases at borders. We propose a pre-screening method that identifies uncorrelated variables prior to estimating the factor model. These uncorrelated variables are then removed from the dataset before factor analysis is conducted. This two-step pre-screening procedure first orders variables according to some measure of their correlation with other variables, then uses likelihood-based stopping criteria to determine which variables in the ordering should be estimated as uncorrelated. Ordering methods considered include measures of absolute and maximum correlation, the squared multiple correlation, a likelihood ratio, and a new technique that forces measurement error into the likelihood in order to identify unimportant variables. Various forms of a correlation matrix are also considered in order to best separate correlated and uncorrelated variables. Finally, stopping criteria include information criteria approaches as well as likelihood ratios, with one possible adaptation focused on controlling the false selection rate. Simulation studies show that some of these new pre-screening methods are competitive, though not uniformly superior to, the traditional approaches to variable selection in factor analysis.