Evolutionary Artificial Neural Networks

Much of my current research work is focussed on using biologically-inspired algorithms to model the development of neuron-to-neuron connectivity. Specifically, this is the simulation of the growth and pruning (optimisation) of artificial axons and dendrites in a 3D environment.

My work occupies a mid-ground between truly biologically plausible models and traditional artificial neural networks (ANNs). Biologically plausible models can be extremely computationally expensive and impractical to use with evolutionary algorithms (EAs). Similarly ANN models which are able to modify their architectures, either through constructive or evolutionary algorithms, tend to be restricted in the range of architectures that they create and lack the dynamical behaviour of biological neural systems. Therefore, I have been using simple, cellular automata-like rules to mimic development, in an effort to reduce the computational load whilst being able to generate a range of generic neural structures. The major questions that I am exploring are can simple developmental/generative rules capture the complexity of development and if so, how detailed should the rules be?

The developmental rules that I am particularly interested are those mimicking growth cone migration, branching, automatic regulation of growth and pruning of neural components (neurons, synapses and dendrites). The rules are governed by parameters which control the scope of potential developmental programmes and thereby the range of neuron and network architecture which can be created. An important feature of the developmental simulator is that the growing neural components interact with each other, mediated by a simplified chemical environment. This introduces competition and co-operation between neurons as their axons and dendrites develop.

My model of neural development is able to create a wide variety of neural morphologies by varying the parameters which control and modify the developmental rules. Designing neural structures to order therefore becomes the search for optimal sets of parameters and I use the genetic algorithm as a tool to do this. Evolution is thereby used to `tune' the developmental process rather than being the critical component which determines network architectures. The emphasis is placed on the genotype-to-phenotype mapping and the interactions which occur during the developmental process, rather than the specific EA employed.

Applying EAs to large developmental models can however be problematic. Complex developmental models can have large, rugged search spaces which can be difficult for EAs to effectively traverse. Evolution can stall in the initial generations and get side-tracked into local minimas. Using the developmental simulator, I have been investigating ways to overcome such problems by evolving developmental programmes in stages. Rather than presenting the EA with the complete developmental programme, development is divided into sets of rules and these sets are evolved in stages. Once optimal sets of parameters have been identified for one evolutionary stage, they are then incorporated in subsequent stages. Evolution therefore incrementally builds upon previous results and the search space is traversed in an orderly manner.

Staged evolution seeks to mimic aspects of speciation in Nature, whereby the complexity of architectures that a population of networks can achieve is determined by its developmental programme. As the complexity of the developmental programme increases, so do the potential performances of evolved networks. This staged evolution scheme differs from other genome enlarging schemes (e.g. SAGA) as it is the complexity of the developmental programme that is increasing, not simply the number of neurons in the network and their connectivity schemes.

Currently I am exploring the development of an artificial retina containing neural structures with on-off centre responses. Evolution begins by operating on symmetrically arranged retinas which can be solved using developmental programmes of relatively minimal complexity. Perturbing the positions of the neurons makes the problem more difficult for the developmental programmes, such that to improve performance, more complex developmental rules are incorporated into the evolutionary procedure in stages.

I am interested in the intrinsic self-organisation observed in biological development during the genotype-to-phenotype mapping. During the development of neural systems, self-organisation occurs across many different levels (molecular, genetic, neuron and network) and is an important phenomenon as it permits development to be robust to `errors', such as perturbations in the positioning and connectivity of neurons.

I am exploring the hypothesis that the intrinsic self-organisation which occurs during gene expression, permits evolution to discover sets of developmental parameters which satisfy a given problem, rather than single, unique solutions. Sets of developmental parameters can demonstrate a higher degree of robustness in the presence of noise, which is desirable when trying to engineer solutions.