Optimization Course SS 13 U Stuttgart


Optimization is one of the most fundamental tools of modern sciences. Many phenomena -- be it in computer science, artificial intelligence, logistics, physics, finance, or even psychology and neuroscience -- are typically described in terms of optimality principles. The reason is that it is often easier to describe or design an optimality principle or cost function rather than the system itself. However, if systems are described in terms of optimality principles, the computational problem of optimization becomes central to all these sciences.


This lecture aims give an overview and introdution to various approaches to optimization together with practical experience in the exercises. The focus will be on continuous optimization problems and we will cover methods ranging from standard convex optimization and gradient methods to non-linear black box problems (evolutionary algorithms) and optimal global optimization. Students will learn to identify, mathematically formalize, and derive algorithmic solutions to optimization problems as they occur in nearly all disciplines. A preliminary list of topics is:

  • gradient methods, log-barrier, conjugate gradients, Rprop
  • constraints, KKT, primal/dual
  • Linear Programming, simplex algorithm
  • (sequential) Quadratic Programming
  • Markov Chain Monte Carlo methods
  • 2nd order methods, (Gauss-)Newton, (L)BFGS
  • blackbox stochastic search, including a discussion of evolutionary algorithms
Students should bring basic knowledge of linear algebra and analysis.
Organization
  • This is the central website of the lecture. Link to slides, exercise sheets, announcements, etc will all be posted here.
  • More information to come
Schedule, slides & exercises
date topics slides exercises
(due on 'date'+5)
18.04. Intro 01-introduction
02-gradientMethods 		e01-gradientMethods1
25.04. Gradient Methods & Constrained Optimization 03-constrainedOpt e02-penaltyAndBarrier
16.05. Constrained Optimization (cont.) e03-lagrangian
24.05. [Pfingsten]
30.05. [Frohnleichnam] [no exercise on 4.6.]
06.06. 2nd Order Optimization Methods 04-secondOrderOpt e04-GaussNewton
13.06. Convex Problems 05-convexProblems e05-convexOpt
20.06. Blackbox Optimization 06-blackBoxOpt
27.06. -- cancelled -- e06-stochasticSearch
04.07. Global Optimization 07-globalOptimization e07-globalOptim
11.07. Summary 13-Optimization-script e07-globalOptim (Exercise 2 now)
../data/gp01pred.m ../data/test.m
Literature

Recent Posts