Optimizers   

Algorithms

• Ipopt
  • Algorithm paper is here and here
• CFSQP
• lmdif
• LAPACK - supports linear least squares with no bound constraints

Notes

• Eran's summary of comparing paramopt, LBFGSB, IPOPT, and CFSQP: optimizer_roundup.txt, and more notes:  optimizer_notes.txt
• Active set methods: combinatorially try to identify the active set
• Also, from this paper:
  On problems with few degrees of freedom, the number of active constraints remains small and active set solvers can identify them quickly with little computational effort. With interior point methods, however, one pays a 'fixed price' for the factorization of the KKT matrix regardless of the number of active constraints. As a result, this 'fixed price' can overwhelm the expense of the active set solver. This is especially true for nonlinear programs since 'warm start' strategies (where a good guess for an active set is known) are widely used for active set solvers. On the other hand, warm starts still present many open questions for interior point methods.
• We now use lapack for RRA by solving it without bounds on the controls and then checking if the solved controls fell within the required bounds. Most of the time this is the case, and if it isn't we fall back on IPOPT.
• I thought we could try this for CMC, too, but then realized that the muscle forces often did achieve their lower bounds (for inactive muscles) so lapack could not be used in the end.

Links

• Various links I collected as I was looking at the optimizers... may not be too useful
• Local optimization software
• Nonlinear programming FAQ
• Quadatic Programming
• Specific software
  • Asynchronous Parallel Pattern Search
  • OPT++
  • CONDOR
  • WNLIB
  • Software by Prof. Schittkowski