EOSC 555B: Nonlinear Inverse Theory

Lecture notes for EOSC 555B: Nonlinear Inverse Theory, taken at the University of British Columbia. Portions of the template code originate from the course instructor, Prof. Eldad Haber, while I have added my own numerical experiments and examples explored as part of self-study on the topics.

Lecture 11

Score matching has been used in machine learning to estimate the parameters of a probability distribution from empirical samples. This lecture introduces the concept of score matching and its connection to diffusion models and generative AI.

Nov 25, 2024
10 min

Lecture 10

In this mathematical lecture, some of the foundational principles of Bayesian inverse problems and their statistical interpretation are discussed. A set of computational tools are shown that aid in finding the solution to some of these problems.

Nov 22, 2024
18 min

Project Work 1

Geomagnetic inversion is a common problem in geophysics where the goal is to recover a 3D distribution of physical properties of a section of Earth using magnetic field measurements taken over a 2D surface above ground. The problem is ill-posed and requires advanced techniques to recover a model that matches the true distribution. In this project, a series of advanced techniques are applied to attempt to recover the true distribution.

Nov 15, 2024
34 min

Lecture 9

Neural networks have revolutionized the field of machine learning, but how exactly do they work? In this lecture, we will explore the basic structure of these models from a mathematical perspective. We will also discuss the role of regularization and priors in solving inverse problems.

Nov 5, 2024
17 min

Lecture 8

Inverse problems often have multiple or infinite solutions. Regularization and priors can be used to constrain the solution space and find a unique solution. This lecture covers the role of regularization and priors in solving inverse problems.

Nov 3, 2024
10 min

Lecture 7

Gaussian homotopy is a technique that effectively spreads the gradient information of a non-convex function outward, helping the optimizer escape local minima.

Oct 22, 2024
16 min

Lecture 6

Automatic differentiation is a powerful tool for solving optimization problems that can be used to automate the process of Gauss-Newton optimization. Here we put together an implementation of the Gauss-Newton method using PyTorch.

Oct 8, 2024
22 min

Lecture 5

The Gauss-Newton method is a powerful optimization technique for solving non-linear least squares problems of the form \(\min_{p} \frac{1}{2}\|F(p)-d\|^2\). In this lecture, a derivation of the method is presented along with a comparison to Newton’s method. Finally an algorithm for solving the problem is given.

Sep 25, 2024
15 min

Lecture 4

Tikhonov regularization is a common technique used in inverse theory to stabilize ill-posed problems. In this lecture, we derive the Tikhonov regularization technique, and we also look at a least squares solution that does not require computing the full SVD of the matrix \(A\), using the conjugate gradient method.

Sep 20, 2024
8 min

Lecture 3

In continuation of Lecture 2, we now look at an alternative approach to image denoising using gradient descent and early stopping. We will derive the least squares gradient descent algorithm and analyze it as an ordinary differential equation.

Sep 17, 2024
22 min

Lecture 2

Image denoising and deblurring are important techniques in signal processing and recovery. In this coding exercise, we explore the application of least squares, the SVD, and the pseudoinverse to denoise and deblur images.

Sep 15, 2024
22 min

Lecture 1

Inverse theory has broad applications across many scientific disciplines. This lecture introduces the concepts of least squares and the singular value decomposition (SVD) as a foundation for understanding inverse theory. We then use these properties to analyze the stability and conditioning of linear systems, solving inverse problems with the pseudoinverse and machine learning techniques.

Sep 14, 2024
11 min
No matching items