Sparse PCA. The most intuitive and straight-forward approach of obtaining sparse PC solutions is to treat PC loadings elements smaller than some threshold value as zero. How- ever, this approach has been demonstrated as misleading because the importance of variables is not determined by the magnitude of variables (Xxxxxx and Xxxxxxxx, 1995a). Another straight-forward approach is to impose sparsity constraints on PC load- ings. Following this direction, Xxxxxxxx et al. (2003a) proposed a LASSO (least abso- lute shrinkage and selection operator) (Tibshirani, 1996) based PCA method, which is named SCoTLASS (Simplified Component Technique-LASSO). This method imposes LASSO constraints on PC loadings, which sacrifices explained variance to achieve sparsity and improve interpretability. We follow the notation used in 1.2 and XXxX- LASS finds αr so that max ||αr||=1 r s subject to αTαr = 0, ∀s < r, r = 2, . . . , q min(p, n − 1), k=1 and Σp |αrk| ≤ γ where αrk is the k−th element of the rth PC loading and γ is some tuning pa- rameter. Although SCoTLASS is easy to understand and has been approved in Xxxxxxxx et al. (2003a) to be effective, there is no efficient algorithm to solve 1.5. According to Zou et al. (2006), the algorithm proposed in Xxxxxxxx et al. (2003a) is expensive and sometimes make cross-validation for selecting optimal tuning parameter infeasible. Thus Zou et al. (2006) proposes SPCA (Sparse Principal Component Analysis). In SPCA, they formulate PCA into a regression-type optimization problem and impose lasso or elastic net (Zou and Xxxxxx, 2005a) constraint on the regression coefficients. Zou et al. (2006) proves that PCA can be transformed into the following regression- type formulation: Σ (Aˆ, Bˆ) = arg min A,B n i=1 ||xi − ABTxi||2 + λ Σr ||βj||2 (1.6) j=1 subject to ATA = Ir×r Where Ap×r = [α1, . . . , αr] and Bp×k = [β1, . . . , βr]. λ is a tuning parameter. βi is proportional to the i−th principal component loadings. Using formalation1.6, SPCA consider the following optimization problem: Σ (Aˆ, Bˆ) = arg min A,B n i=1 ||xi − ABTxi||2 + λ Σr ||βj||2 + Σr λ1,j||βj||1 (1.7) j=1 j=1 subject to ATA = Ir×r Here λ is the tuning parameter for all r components and λ1,j allows for different penalization on different components. Besides theoretical proof of the methods, Zou et al. (2006) also proposes efficient algorithms to solve 1.7 which is included in R package elasticnet. Besides SPCA, another popular sparse principal component method, called SPC, is published...
