A model with an intrinsic property of learning higher order correlations

Abstract

A neural network model that can learn higher order correlations within the input data without suffering from the combinatorial explosion problem is introduced. The number of parameters scales as (M) over bar X N, where (M) over bar is the number such that no higher order network with less than (M) over bar higher order terms can implement the same input data set and N is the dimensionality of the input vectors. In order to have better generalization, the model was designed to realize a supervised learning such that after learning, output for any input vector is the same as the output of a higher order network that implements the same input data set using (M) over bar number of higher order terms. Unlike the case in product units, the local minima problem does not pose itself as a severe problem in the model. Simulation results for some problems are presented and the results are compared with the results of a multilayer feedforward network. It is observed that the model can generalize better than the multilayer feedforward network. (C) 2001 Elsevier Science Ltd. All rights reserved.