Classifier training - Feature selection Sample Clauses

Classifier training - Feature selection. ‌ Model parameters are tuned using grid search with 4-fold cross validation. Class imbalance was handled using smote, down or up subsampling, performed inside the cv resampling. Downsampling was chosen for the final models. A number of classifiers was initially tested (random forest, xgboost, partial least squares (pls), naïve bayes, etc.). Random forest, consistently showed good performance and was finally chosen for the development of the models. Feature selection was performed based on recursive feature elimination (RFE). Random forest specific variable importance was utilized during the RFE procedure. For random forest classifiers, there is a plateau of good performance after a specific number of features. The optimal number of predictors is chosen by taken into account this performance profile. We pick a small subset size that yields a performance score within tolerance of the best (but more complex) model, without sacrificing too much performance. It is noted that feature selection is not a deterministic procedure and results may vary due to differences in resampling, splitting and, overall, due to the stochastic nature of the algorithms used. RFE is repeated a number of times to evaluate the variability of the selected features. The most frequently selected features are finally chosen. The caret package of R is used.