On the degrees of freedom in richly parameterized models

Using richly parameterised models for small datasetscan be justified from a theoretical point of viewaccording to some results due to Bartlett which show that thegeneralization performance of a multi layer perceptron (MLP)depends more on the L1 norm of the weightsbetween the hidden and the output layer rather than on thenumber of parameters in the model.In this paper we investigate the problem of measuring the generalizationperformance and the complexity of richly parameterised procedures and,drawing on linear model theory, we propose a different notion of degrees of freedomto neural networks and other projection tools. This notion is compatible withsimilar ideas long associated with smoothers based models(like projection pursuit regression) and can be interpreted using the projectiontheory of linear models and showing some geometrical properties of neural networks.Results in this study lead to corrections insome goodness-of-fit statistics like AIC, BIC/SBC:the number of degrees of freedom in theseindexes are set equal to the dimension p of the projection spaceintrinsically found by the mapping function.An empirical study is presented in order toillustrate the behavior of the valuesof some selection model criteria.