Optimizers

Algorithms

Eran's summary of comparing paramopt, LBFGSB, IPOPT, and CFSQP: optimizer_roundup.txt, and more notes: optimizer_notes.txt
Active set methods: combinatorially try to identify the active set
Also, from this paper:
On problems with few degrees of freedom, the number of active constraints remains small and active set solvers can identify them quickly with little computational effort. With interior point methods, however, one pays a 'fixed price' for the factorization of the KKT matrix regardless of the number of active constraints. As a result, this 'fixed price' can overwhelm the expense of the active set solver. This is especially true for nonlinear programs since 'warm start' strategies (where a good guess for an active set is known) are widely used for active set solvers. On the other hand, warm starts still present many open questions for interior point methods.
We now use lapack for RRA by solving it without bounds on the controls and then checking if the solved controls fell within the required bounds. Most of the time this is the case, and if it isn't we fall back on IPOPT.
I thought we could try this for CMC, too, but then realized that the muscle forces often did achieve their lower bounds (for inactive muscles) so lapack could not be used in the end.