Applied Optimal Control
ECE-DK905 · Winter semester
Electrical & Computer Engineering Dept., University of Patras
About the course
A practical, implementation-first course on optimal control for robotics. It starts from the dynamics and discretization of controlled systems and the numerical optimization they rely on — Newton’s method, line search, constrained optimization, augmented Lagrangian methods, ADMM and quadratic programming.
On this foundation it develops LQR, Bellman’s principle and dynamic programming, convex MPC, direct collocation for nonlinear trajectory optimization, rotations and quaternions for optimization on manifolds, differential dynamic programming and iLQR, and contact-rich optimal control for legged and multi-limbed robots.
Course organization
- Theory-focused lectures with live coding, and a practical laboratory.
- Laboratory: students build a real pendulum and implement PID, LQR, TVLQR and linear MPC controllers on it.
Syllabus
- Lec. 1Dynamics, Discretization and Integrators
- Lec. 2Root Finding and Optimization via Newton’s Method
- Lec. 3Constrained Optimization Methods
- Lec. 4Deterministic Optimal Control & LQR
- Lec. 5Model Predictive Control & Nonlinear Trajectory Optimization
- Lec. 6Matrix Lie Groups and Poses
- Lec. 7Optimization and Optimal Control on Manifolds
- Lec. 8Differential Dynamic Programming
- Lec. 9Optimal Control for Robots with Multiple Limbs/End-Effectors
- Lec. 10Optimal Control for Complex Robots
Lecture notes
The notes are written per topic, so one document often covers several lectures.
Lectures & materials, week by week
Slides and notes for each lecture, grouped by teaching week.
Week 1
Lecture 1
Dynamics, Discretization and Integrators
- Dynamics Review: Differential Equations, Control Affine Systems, Manipulator Equation
- Discretization of Dynamical Systems (explicit): Euler, Semi-Implicit Euler, RK4
- Discretization of Linear Systems: analytical solution
- Examples: stability of linear and non-linear systems
Slides
Shared notes · Lectures 1–3: Part I: A Careful Optimization Primer
Notes
Week 2
Lecture 2
Root Finding and Optimization via Newton’s Method
- Notation, Jacobians, Derivatives, Gradients, Hessians
- Taylor Series for Controlled Systems (aka decomposition of state and action variables)
- Root finding and fixed point problems
- Newton's Method for root finding: examples on implicit integrators
- Newton's Method for optimization (root finding on gradient): simple, damping (regularization)
- Line-Search: General and Armijo rule
- Constrained Optimization with Equality Constraints: first order necessary conditions, Lagrangian (with multipliers), Gauss-Newton
Slides
Shared notes · Lectures 1–3: Part I: A Careful Optimization Primer
Notes
Week 3
Lecture 3
Constrained Optimization Methods
- Constrained Optimization: Duality
- Constrained Optimization with Inequality Constraints: first order necessary conditions
- Augmented Lagrangian Method
- Merit Functions
- Constrained Optimization with Equality and Inequality Constraints
- Alternating Direction Method of Multipliers (ADMM)
- Quadratic Programming (QP)
- Quick overview of popular optimization methods: Interior Point Methods, Sequential Quadratic Programming, ...
Slides
Shared notes · Lectures 1–3: Part I: A Careful Optimization Primer
Notes
Week 4
Lecture 4
Deterministic Optimal Control & LQR
- Deterministic Optimal Control Problem: Continuous and Discrete
- Pontryagin’s Minimum Principle (no derivation, intuitive explanation)
- The LQR problem
- LQR via shooting
- LQR as a QP problem
- LQR via the Riccati recursion
- Infinite Horizon LQR
- Controllability
Slides
Shared notes · Lectures 4 & 5: Part II: Linear Optimal Control
Notes
Week 5
Lecture 5
Model Predictive Control & Nonlinear Trajectory Optimization
- Bellman's Principle
- Dynamic Programming (DP): generic formulation for time-variant systems (aka one value function per step k)
- LQR from Bellman (aka DP on the LQR problem)
- Connections between Lagrange Multipliers and the Value Function
- Convex Model Predictive Control
- Linearization over a fixed point + LQR
- Direct collocation: non linear trajectory optimization
- Cubic Splines
- Quadrature (trapezoidal + Hermite-Simpson)
- Examples with ipopt
Slides
Shared notes · Lectures 4 & 5: Part II: Linear Optimal Control
Notes
Week 6
Lecture 6
Matrix Lie Groups and Poses
- Orientation Representations
- Rotation Matrix
- Direction Cosine Method/Matrix (DCM)
- Skew Symmetric Matrices
- Integration of rotation matrices
- Axis-Angle
- Euler angles (and why to avoid them)
- Quaternions
- Mapping between angular velocity and derivatives of parameters of representations
- Representation-Free Parameterization
Slides
Shared notes · Lectures 6–8: Part III: Nonlinear Optimal Control
Notes
Week 7
Lecture 7
Optimization and Optimal Control on Manifolds
- Planning with attitude: quaternions done right
- Rodrigues Parameters and Quaternions
- Quaternions: orientation/attitude Jacobian
- Taylor Series with Quaternions
- Unconstrained Minimization with Quaternions
- Linearization over a trajectory
- Linearizing Quaternions
- 3D Quadrotor code example
Slides
Shared notes · Lectures 6–8: Part III: Nonlinear Optimal Control
Notes
Week 8
Lecture 8
Differential Dynamic Programming
- Kronecker product, vectorization (flattening) and the "vec-trick"
- Action-Value ("Q") Function
- Differential Dynamic Programming (DDP) derivation via second order Taylor expansion on the Q-function
- DDP pseudocode/algorithm
- ILQR
- Cartpole code example
Slides
Shared notes · Lectures 6–8: Part III: Nonlinear Optimal Control
Notes
Week 9
Lecture 9
Optimal Control for Robots with Multiple Limbs/End-Effectors
- Control for Robots with Multiple Limbs
- Contact Modelling (contact modes, contact forces, friction, etc..)
- Controlling with Contacts: Friction Cones
- Hybrid/Event-based methods: Jump functions
- Implicit/Time-stepping methods: signed distance functions
- Single Rigid Body Dynamics (SRBD) Model
- Approximating friction cones with pyramids
- Gait Sequence/Scheduling
- Monopod without Orientation Code Example
Slides
Shared notes · Lectures 9 & 10: Part IV: Complex Robots
Notes
Week 10
Lecture 10
Optimal Control for Complex Robots
- Single Rigid Body Dynamics (SRBD) model, revisited
- Approximating friction cones with pyramids, revisited
- Gait sequencing/scheduling, revisited
- Full Monopod Code Example
- TALOS floating-base manifold multiple-shooting and jump-map notebooks (kinematic and full-dynamics examples, with local URDF/mesh replay)
- Optimal Control in Generalized Reduced Coordinates
- Derivatives via the Pinocchio library
- Jump Maps in Generalized Reduced Coordinates
- 7DoF Manipulator Code Example
- Full limb Monopod Code Example
- Control pipeline of Complex Robots
Slides
Shared notes · Lectures 9 & 10: Part IV: Complex Robots
Notes