EOSC 555B: Nonlinear Inverse Theory

Lecture notes for EOSC 555B: Nonlinear Inverse Theory taken at the University of British Columbia. Portions of template code originate from the course instrcutor Prof. Eldad Haber, while I’ve added some of my own numerical experiments and examples explored as part of self-study on the topics.

Lecture 11

Score Matching and Diffusion

Score matching has been used in machine learning to estimate the parameters of a probability distribution from emiprical samples. This lecture will introduce the concept of score matching and its connection to diffusion models and generative AI.

Lecture 10

MAP, MLE, and Score Function

In this mathematical lecture, some of the foundational principles of Bayesian inverse problems and their statistical interpretation are discussed. A set of computational tools are shown that aid in finding the solution to some of these problems.

Project Work 1

Applications of Machine Learning in Geophysics

Geomagnetic inversions are a common problem in geophysics where the goal is to recover a 3D distribution of the conductivity of a section of Earth using magnetic field measurements taken over a 2D surface above ground. The problem is ill-posed and requires advanced techniques to recover data that matches the true distribution. In this project, a series of advanced techniques are used to try and recover true distributions.

Lecture 9

Machine Learning and Neural Networks

Neural networks have revolutionized the field of machine learning, but how exactly do they work? In this lecture, we will explore the basic structure of these models from a mathematical perspective. We will also discuss the role of regularization and priors in solving inverse problems.

Lecture 8

Regularization and Priors in Inverse Problems

Inverse problems often have multiple or infinite solutions. Regularization and priors can be used to constrain the solution space and find a unique solution. This lecture covers the role of regularization and priors in solving inverse problems.

Lecture 7

Homotopy Optimization

Gaussian homotopy is a technique that can be used to effectively broadcast the gradient of a non-convex function outward to help scape local minima.

Lecture 6

Autodiff and Coding Gauss-Newton for Least Squares

Automatic differentiation is a powerful tool for solving optimization problems that can be used to automate the process of Gauss-Newton optimization. Here we put together an implementation of the Gauss-Newton method using PyTorch.

Lecture 5

Gauss Newton for Non-Linear Least Squares Optimization

The Gauss-Newton method is a powerful optimization technique for solving non-linear least squares problems of the form \(\min_{p} \frac{1}{2}\|F(p)-d\|^2\). In this lecture, a derivation of the method is presented along with a comparison to Newton’s method. Finally an algorithm for solving the problem is given.

Lecture 4

Regularization and the Conjugate Gradient Methods

Tikhonov regularization is a common technique used in inverse theory to stabilize ill-posed problems. In this lecture, we derive the Tikhonov regularization technique, we also have a look at a least squares solution that does not require the computation of the full SVD of the matrix \(A\), using the conjugate gradient method.

Lecture 3

Image Denoising with Gradient Descent and Early Stopping

In continuation of Lecture 2, we now look at an alternative approach to image denoising using gradient descent and early stopping. We will derive the least squares gradient descent algorithm and analyze it as an ordinary differential equation.

Lecture 2

Image Denoising with SVD

Image denoising and deblurring are important techniques in signal processing and recovery. I this coding exercise, we will explore the application of least squares, SVD, and the pseudoinverse to denoise and deblur images.

Lecture 1

Introduction to Inverse Theory

Inverse theory has broad applications across many scientific disciplines. This lecture introduces the concept of least squares and the singular value decomposition (SVD) as a foundation for understanding inverse theory. We then use these properties to analyse the stability and conditioning of linear systems for solving inverse problems using the pseudoinverse and ML techniques.