User:Jarle Pahr/SciPy

Notes on the SciPy Python library:

http://www.scipy.org/

http://scipy-lectures.github.io/

http://oneau.wordpress.com/2011/02/28/simple-statistics-with-scipy/

Testing:

import scipy as sci
sci.test()

Installation files

http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy-stack

Linalg

http://docs.scipy.org/doc/scipy/reference/linalg.html

Optimization

http://scicomp.stackexchange.com/questions/83/is-there-a-high-quality-nonlinear-programming-solver-for-python

Tutorial: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial

http://stackoverflow.com/questions/49926/open-source-alternative-to-matlabs-fmincon-function

For comparison see http://www.mathworks.se/help/optim/ug/fmincon.html

See also http://scicomp.stackexchange.com/questions/83/is-there-a-high-quality-nonlinear-programming-solver-for-python and http://openopt.org/SciPy

See also: http://scipy-lectures.github.io/advanced/mathematical_optimization/

Constrained minimization of multivariate scalar functions (minimize): http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp

Functions

minimize: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize

• General interface to several methods for minimization of multi-variate scalar function.

fmin: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin.html#scipy.optimize.fmin

• Non-linear optimization
• Uses the heuristic Nelder-Mead simplex method: http://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method
• (Don't confuse with Danzig's simplex method for linear programming)
• Equivalent to MATLABs fminsearch: http://stackoverflow.com/questions/19070943/numpy-scipy-analog-of-matlabs-fminsearch

Cobyla: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cobyla.html#scipy.optimize.fmin_cobyla

• Constrained optimization with inequality constraints
• Variable bounds and equality constraints not explicitly supporte (must be implemented as inequality constraints).

Sequential Least Squares Programming (SLSQP):

• Nonlinear optimization method for problems where the objective function and constraints are twice differentiable.
• Allows variable bounds and equality/inequality constraints.
• fmin_slsqp: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html
• Seems more similar to MATLABs fmincon than optimize.fmin
• Tutorial: docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp
• See also http://www.pyopt.org/reference/optimizers.slsqp.html
• See also: http://scipy-user.10969.n7.nabble.com/using-scipy-optimize-fmin-slsqp-and-setting-bounds-None-None-td14134.html

Global solvers:

Brute: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brute.html#scipy.optimize.brute

scipy.optimize.fmin_ncg

Unconstrained optimization by Newton-Conjugate Gradient(NCG) method.

scipy.optimize.fmin_tnc

Truncated Newton-CG method. Allows variable bounds. Does not support equality/inequality constraints.