Regression Model Selection using Genetic Algorithms

The selection of independent variables in a regression model is often a challenging problem. Ideally, one would like to obtain the most adequate regression model. This task can be tackled with techniques such as expert based selection, stepwise regression and stochastic search heuristics, such as genetic algorithms (GA). In this study, we investigate the performance of two GAs for regressors selection (GARS) and for regressors selection with transformation of the regressors (GARST). We compare the performance with stepwise regression for the “Fat Measurement” and the “Cholesterol Measurement” datasets and use the AIC, BIC and SIC statistical criteria to quantify the adequacy of the models. The results for GARS are superior for all statistical criteria compared to both forward and backward stepwise regression, but not always when R2 and RMSE statistics are considered. GARST turns out to be even better compared to GARS as variable transformations help to improve results further. Moreover, the type of transformations revealed the relationships between dependent and independent variables.