Numerical Solution of Implicitly Constrained Optimization Problems

Matthias Heinkenschloss
Department of Computational and Applied Mathematics
Rice University

CAAM Technical Report TR08-05 (rev. June 2012, Jan 2013)


Many applications require the minimization of a smooth function f: Rn → R whose evaluation requires the solution of a system of nonlinear equations. This system represents a numerical simulation that must be run to evaluate f. We refer to this system of nonlinear equations as an implicit constraint.

In theory f can be minimized using the steepest descent method or Newton-type methods for unconstrained minimization. However, for the practical application of derivative based methods for the minimization of f one has to deal with many interesting issues that arise out of the presence of the system of nonlinear equations that must be solved to evaluate f.

This article studies some of these issues, ranging from sensitivity and adjoint techniques for derivative computation to implementation issues in Newton-type methods. A discretized optimal control problem governed by the unsteady Burgers equation is used to illustrate the ideas.

The material in this article is accessible to anyone with knowledge of Newton-type methods for finite dimensional unconstrained optimization. However, many of the concepts discussed in this article extend to and are used in areas such as optimal control and PDE constrained optimization.

Keywords. Unconstrained minimization, implicit constraints, adjoints, sensitivities, Newton method, nonlinear programming, optimal control, Burgers equation.

Matlab codes