Notes on the SciPy Python library:
import scipy as sci sci.test()
For comparison see http://www.mathworks.se/help/optim/ug/fmincon.html
Constrained minimization of multivariate scalar functions (minimize): http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp
- General interface to several methods for minimization of multi-variate scalar function.
- Unconstrained,non-linear optimization
- Uses the heuristic Nelder-Mead simplex method: http://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method
- (Don't confuse with Danzig's simplex method for linear programming)
- Equivalent to MATLABs fminsearch: http://stackoverflow.com/questions/19070943/numpy-scipy-analog-of-matlabs-fminsearch
- Unconstrained, non-linear optimization using Powell's conjugate direction method
- Unconstrained, non-linear optimization using a conjugate gradient algorithm.
Constrained Optimization BY Linear Approximation (COBYLA): http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cobyla.html#scipy.optimize.fmin_cobyla
- Constrained non-linear optimization with inequality constraints
- Variable bounds and equality constraints not explicitly supported (must be implemented as inequality constraints).
Advances in Optimization and Numerical Analysis Mathematics and Its Applications Volume 275, 1994, pp 51-67 A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation. M. J. D. Powell.
Sequential Least Squares Programming (SLSQP):
- Nonlinear optimization method for problems where the objective function and constraints are twice differentiable.
- Allows variable bounds and equality/inequality constraints.
- fmin_slsqp: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html
- Seems more similar to MATLABs fmincon than optimize.fmin
- Tutorial: docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp
- See also http://www.pyopt.org/reference/optimizers.slsqp.html
- See also: http://scipy-user.10969.n7.nabble.com/using-scipy-optimize-fmin-slsqp-and-setting-bounds-None-None-td14134.html
- Unconstrained optimization by Newton-Conjugate Gradient(NCG) method.
- Truncated Newton-CG method. Allows variable bounds. Does not support equality/inequality constraints.
- Unconstrained, non-linear optimization using the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm.
- Non-linear optimization with bound constraints using the limited-memory BFGS with boundaries (L-BFGS-B) algorithm.
- Simulated annealing. Non-linear optimization. Supports bound constraints.