CONTENTS
- Calculus & Linear Algebra Prerequisites
- Taylor Expansion (multivariate)
- Quadratic Functions
- Newton-Raphson Method (single variable)
- Convergence issues
- Vector & Matrix norms
- Condition number
- Matrix Factorizations (LU, QR, Choleski)
- Matrix Updates and decompositions
- Sherman - Morrison formula
- The general optimization problem
- Global case
- Local case
- Optimality conditions for unconstrained local optimization
- Calculus of variations
- Structure of Algorithms
- Descent directions
- Line search algorithms
- Trust region algorithms
- The Dog-Leg technique
- Conjugate Directions
- Conjugate Gradient Methods
- Newton's Method
- Modified Newton's methods
- Quasi - Newton methods
- Rank-1, & Rank-2 updates
- Methods for Sums of Squares
- Gauss-Newton method
- Levenberg-Marquardt
- Hybrid methods
- Introduction to Optimization with constraints
- Lagrange multipliers
- KKT conditions
- Penalty functions
- Barrier functions
- Convex Quadratic Programming
- Bound constraints