Artificial Intelligence II
A second course in artificial intelligence focused on modern machine learning. It begins with the foundations of learning — loss functions, regularization, back-propagation and neural networks — and the optimization that makes training possible, from Newton and Gauss–Newton methods to stochastic gradient descent, momentum and Adam.
It then covers neural networks for computer vision (CNNs, attention and transformers), uncertainty quantification with Bayesian linear regression and Gaussian processes, learning probability distributions (GMMs, EM, ensembles), latent representations (PCA, factor analysis, autoencoders, VAEs), generative models such as flow matching and diffusion, and zero-order optimization: evolutionary strategies, genetic algorithms and swarm intelligence.
- Lec. 1Introduction to Machine Learning
- Lec. 2Optimization Basics
- Lec. 3Stochastic Gradient Descent and Training
- Lec. 4NNs for Computer Vision (Part I)
- Lec. 5NNs for Computer Vision (Part II)
- Lec. 6Uncertainty Quantification and Non-Parametric Models
- Lec. 7Learning Probability Distributions
- Lec. 8Learning Latent Representations
- Lec. 9Generative Models
- Lec. 10Zero-Order Optimization (Part I)
- Lec. 11Zero-Order Optimization (Part II)
Slides for each lecture, grouped by teaching week.
- Linear separation, non-linearities
- Regularization
- Loss functions, optimization
- Back propagation
- Neural Networks
- Deep learning generic architecture
- Intro to Optimization
- Sufficient and Necessary Conditions
- Newton's Method for Optimization
- Line Search
- Non-linear least squares (Gauss-Newton)
- Practical Considerations
- Why second-order methods struggle with large networks and datasets
- First-order optimization methods
- Stochastic gradient descent, momentum, Adam
- Standardization, normalization, cross-validation
- Metrics for hyper-parameter optimization
- Convolution, Image convolution
- CNNs in the image domain
- Pooling, zero padding, cost functions
- CNNs in the 3D domain
- CNN autoencoders
- CNNs in manifold surfaces
- Applications and examples
- Attention in DNN, rationale and impact
- Attention layers modeling
- Vision Transformers
- 3D domain transformers
- Applications and examples
- Uncertainty (epistemic vs aleatoric)
- Bayesian Linear Regression
- Gaussian Processes
- Gaussian Mixture Models and Expectation Maximization
- Neural Networks that output distributions
- Ensemble Neural Networks
- Examples
- PCA
- Factor Analysis
- Auto-Encoders
- VAEs
- Overview: connection to GMMs, Factor Analysis
- Flow Matching
- Diffusion
- Examples
- Black-Box optimization
- Nelder-Mead (Simplex)
- Simulated Annealing
- Genetic Algorithms
- Evolutionary Strategies
- Swarm Intelligence: Particle Swarm Optimization
- Ant Colony Optimization