Vorlesung Numerical Optimization

Content: In this lecture we will learn theories and algorithms of numerical optimization. We study the mathematical structure of typical optimization problems, in order to design efficient and advanced algorithms. Such structure is investigated by accessing the zero-th order (function values), the first order (derivatives), and the second order information (Hessians) about objective functions, as well as by looking into the geometry of constraints. We will discuss constrained and unconstrained optimization problems in continuous spaces, focusing on understanding motivations behind technical details, analyzing convergence rate / algorithm complexity, and applying algorithms. Fundamental concepts such as optimality and duality will be discussed in details, which become popular tools for analysis in many areas including machine learning, data mining, and statistics. The importance of smoothness and convexity will be elaborated, especially in connection to regularization problems in high dimensions. Some advanced topics from non-smooth, large-scale, or matrix optimization will be included if time permits. Homework assignments will be given to check theoretical and practical understanding of techniques. Aims: The aim of this lecture is to provide students with good understanding of fundamental concepts and techniques in optimization, part of them are introduced in the basis module -Praktische Optimierung- but not discussed in an advanced level, so that students can understand, use, and design efficient numerical optimization algorithms for their own research problems. Lectures will be mainly based on the following textbooks: • Numerical Optimization, J. Nocedal S. Wright, 2nd Ed, Springer, 2006 • Introductory Lectures on Convex Optimization, Y. Nesterov, Springer, 2003(4) Some advanced materials will be from: • Nonlinear Programming, D. P. Bertsekas, 2nd Ed., Athena Scientific, 1999 • Convex Optimization, S. Boyd L. Vandenberghe, Cambridge, 2004 Voraussetzungen This lecture is for computer science students and other non-mathematicians, and therefore it will require no special mathematical skill. However, basic understanding of calculus at undergraduate (Bachelor/Diplome) level will be assumed. This lecture is for computer science students and other non-mathematicians, and therefore it will require no special mathematical skill. However, basic understanding of calculus at undergraduate (Bachelor/Diplome) level will be assumed. Technische Universität Dortmund SoSe 2014 Informatik Dr. Lee Sangkyun