Global Optimization Via Neural Networks and DC Programming

Abstract

The ultimate goal of this work is to provide a general global optimization method. Due to the difficulty of the problem, the complete task is divided into several sections than can be summarized as a modeling phase followed by a global optimization phase. Each of the various sections draws from different engineering fields. What this work suggests is an interface and common grounds between these.

The modeling phase of the procedure consists of converting a general problem into a given formulation using a particular type of neural network. The architecture required for the neural network forms a new class: the pseudo multilayer neural network. It is introduced and compared to more classical neural network architectures such as the regular multilayer neural network. However, a key difference between these is that the behavior of the usual multilayer neural network has to be programmed, while an extremely efficient procedure is given here to synthesize the pseudo multilayer neural network. Thereby any initial problem can be systematically converted into a pseudo multilayer network without going through the undesired programming steps such as the backpropagation rule.

The second phase of the work consists of translating the initial global optimization problem into the global minimization of a target function related to the neural network model. Systematic procedures are again given here.

The last phase consists of globally minimizing the target function. This is done via the so-called DC programming technique where DC stands for "Difference of Convex." The pseudo multilayer was created such that it can systematically be converted into a DC formulation, and therefore be compatible with DC programming. A translation procedure to go from the pseudo multilayer neural network to the DC formulation is given. When a DC program is applied to this last formulation, the resulting solution can be directly mapped to the global minimum of the target function previously defined, thereby producing the global solution of the neural network modeling the initial problem. Therefore, the optimal solution of the original problem is known as well.

Interested in reading the entire thesis? (160 pages, 5,073,314 bytes, pdf)

Homepage

Last modified: July 19, 2005 -- © François Cellier