On Combining Artificial Neural Nets

This paper reviews research on combining artificial neural nets, and provides an overview of, and an introduction to, the papers contained in this special issue, and its companion (Connection Science, 9, 1). Two main approaches, ensemble-based, and modular, are identified and considered. An ensemble, or committee, is made up of a set of nets, each of which is a general function approximator. The members of the ensemble are combined in order to obtain better generalization performance than would be achieved by any of the individual nets. The main issues considered here under the heading of ensemble-based approaches are a how to combine the outputs of the ensemble members, b how to create candidate ensemble members and c which methods lead to the most effective ensembles? Under the heading of modular approaches, we begin by considering a divide-and-conquer approach by which a function is automatically decomposed into a number of subfunctions which are treated by specialist modules. Other modular approaches are also identified and considered, for while the divide-and-conquer approach is designed to improve performance, the term modularity can be given a wider interpretation. The broadly defined topic of modularity includes the explicit decomposition of a task based on the designer's understanding, and the exploitation of specialist modules in order to accomplish tasks which could not be performed by a monolithic net.