PUBLIC MARKS from ogrisel with tag optimization

Theoretical results strongly suggest that in order to learn the kind of complicated functions that can represent high-level abstractions (e.g. in vision, language, and other AI-level tasks), one may need "deep architectures", which are composed of multiple levels of non-linear operations (such as in neural nets with many hidden layers). Searching the parameter space of deep architectures is a difficult optimization task, but learning algorithms (e.g. Deep Belief Networks) have recently been proposed to tackle this problem with notable success, beating the state-of-the-art in certain areas. This workshop is intended to bring together researchers interested in the question of deep learning in order to review the current algorithms' principles and successes, but also to identify the challenges, and to formulate promising directions of investigation. Besides the algorithms themselves, there are many fundamental questions that need to be addressed: What would be a good formalization of deep learning? What new ideas could be exploited to make further inroads to that difficult optimization problem? What makes a good high-level representation or abstraction? What type of problem is deep learning appropriate for? The workshop presentation page show selected links to relevant papers (PDF) on the topic.

CVXMOD is a Python-based tool for expressing and solving convex optimization problems. It uses CVXOPT as its solver. It is developed by Jacob Mattingley, as PhD work under Stephen Boyd at Stanford University. CVXMOD is primarily a modeling layer for CVXOPT. While it is possible to use CVXOPT directly, CVXMOD makes it faster and easier to build and solve problems. Advanced users who want to see or manipulate how their problems are being solved should consider using CVXOPT directly. Additional features are being added to CVXMOD beyond just modeling. These are currently experimental. CVXMOD has a similar design philosophy to CVX, a convex optimization modeling language for Matlab®, and uses the principles of disciplined convex programming, as developed by Michael Grant, Stephen Boyd and Yinyu Ye.

# Balanced treatment of the simplex method and interior-point methods. # Efficient source code (in C) for all the algorithms presented in the text. # Thorough discussion of several interior-point methods including primal-dual path-following, affine-scaling, and homogeneous self dual methods. # Extensive coverage of applications including traditional topics such as network flows and game theory as well as less familiar ones such as structural optimization, L^1 regression, and the Markowitz portfolio optimization model. # Over 200 class-tested exercises. # A dynamically expanding collection of exercises.

I chose to write them in pure SSE1 MMX so that they run on the pentium III of your grand mother, and also on my brave athlon-xp, since thoses beast are not SSE2 aware. Intel AMath showed me that the performance gain for using SSE2 for that purpose was not large enough (10%) to consider providing an SSE2 version (but it can be done very quickly). The functions use only the _mm_ intrinsics , there is no inline assembly in the code. Advantage: easier to debug, works out of the box on 64 bit setups, let the compiler choose what should be stored in a register, and what is stored in memory. Inconvenient: some versions of gcc 3.x are badly broken with certain intrinsic functions ( _mm_movehl_ps , _mm_cmpeq_ps etc). Mingw's gcc for example -- beware that the brokeness is dependent on the optimization level. A workaround is provided (inline asm replacement for the braindead intrinsics), it is not nice but robust, and broken compilers are detected by the validation program below.

CVXOPT is a free software package for convex optimization based on the Python programming language. It can be used with the interactive Python interpreter, on the command line by executing Python scripts, or integrated in other software via Python extension modules. Its main purpose is to make the development of software for convex optimization applications straightforward by building on Python's extensive standard library and on the strengths of Python as a high-level programming language.

This book is about convex optimization, a special class of mathematical optimization problems, which includes least-squares and linear programming problems. It is well known that least-squares and linear programming problems have a fairly complete theory, arise in a variety of applications, and can be solved numerically very efficiently. The basic point of this book is that the same can be said for the larger class of convex optimization problems. While the mathematics of convex optimization has been studied for about a century, several related recent developments have stimulated new interest in the topic. The first is the recognition that interior-point methods, developed in the 1980s to solve linear programming problems, can be used to solve convex optimization problems as well. These new methods allow us to solve certain new classes of convex optimization problems, such as semidefinite programs and second-order cone programs, almost as easily as linear programs. The second development is the discovery that convex optimization problems (beyond least-squares and linear programs) are more prevalent in practice than was previously thought. Since 1990 many applications have been discovered in areas such as automatic control systems, estimation and signal processing, communications and networks, electronic circuit design, data analysis and modeling, statistics, and finance. Convex optimization has also found wide application in combinatorial optimization and global optimization, where it is used to fond bounds on the optimal value, as well as approximate solutions. We believe that many other applications of convex optimization are still waiting to be discovered. There are great advantages to recognizing or formulating a problem as a convex optimization problem. The most basic advantage is that the problem can then be solved, very reliably and efficiently, using interior-point methods or other special methods for convex optimization. These solution methods are reliable enough to be embedded in a computer-aided design or analysis tool, or even a real-time reactive or automatic control system. There are also theoretical or conceptual advantages of formulating a problem as a convex optimization problem. The associated dual problem, for example, often has an interesting interpretation in terms of the original problem, and sometimes leads to an efficient or distributed method for solving it. We think that convex optimization is an important enough topic that everyone who uses computational mathematics should know at least a little bit about it. In our opinion, convex optimization is a natural next topic after advanced linear algebra (topics like least-squares, singular values), and linear programming.

EvoGrid is a componentized framework based on the Zope3 interfaces / adapters system to build Evolutionary Algorithms (aka Genetic Algorithms) by pluging python components together.

EvoGrid is a component-based python framework to build Evolutionary Computation-based Machine Learning algorithms sometime also known as Genetic Algorithms The EvoGrid design is inspired by the idea of "replicators" introduced by Richard Dawkins in his book The Selfish Gene. EvoGrid's replicators can evolve through both classical undirected darwinian evolution or through "intelligent" lamarckian evolution or by a combination of both. In this respect, EvoGrid can be considered a Memetic Computational framework.