CAAM 454 / 554
Iterative Methods for Systems of Equations and Unconstrained Optimization

Spring 2020 · Rice University


MAIN PAGE  //  LECTURES   //  PROBLEM SETS


Lectures

Lecture 1
Tues., Jan. 14:
Course overview; examples of linear systems (finite difference discretization of elliptic PDEs).
Reading: Chapter 1 in the CAAM 454/554 Notes.
Matlab: adv_diff_1d.m, adv_diff_2d.m, blktridiag.m, adv_diff_1d_per.m.
Python: AdvDiffPDE1D.ipynb.
Additional reading: Chapter 2 in Y. Saad: Iterative Methods for Sparse Linear Systems and Sections 16.1, 16.2 in J. Nocedal and S. J. Wright, Numerical Optimization.
Lecture 2
Thurs., Jan. 16:
Examples of linear systems (finite difference discretization of elliptic PDEs, data assimilation).
Reading: Chapter 1 in the CAAM 454/554 Notes.
Matlab: ocp_adv_diff_1d.m, data_assim_1d.m, Av.m,Atz.m, AtAv.m.

Lecture 3
Tues., Jan. 21:
Stationary methods; convergence of stationary iterative methods.
Reading: Sections 2.1-2.5 in the CAAM 454/554 Notes.
Matlab: adv_diff_1d_stencil_sor.m, adv_diff_2d_stencil_sor.m.

Lecture 4
Thurs., Jan 23:
Convergence of stationary iterative methods; convergence of Jacobi, Gauss-Seidel, SOR.
Reading: Sections 2.5-2.7 in the CAAM 454/554 Notes.

Lecture 5
Fri., Jan. 24
MEL 251
:
Convergence of Jacobi, Gauss-Seidel, SOR.
Reading: Sections 2.5-2.7 in the CAAM 454/554 Notes.
Matlab: adv_diff_1d_stencil_sor.m, adv_diff_2d_stencil_sor.m.

Tues., Jan. 28: No class.

Tues., Jan. 30: No class.

Lecture 6
Tues., Feb. 4:
The gradient and the Conjugate Gradient method.
Reading: Sections 3.2, 3.3 in the CAAM 454/554 Notes.
Additional Reading: Section 5.1 in Nocedal and Wright, Numerical Optimization, Lecture 38 in Trefethen and Bau, Numerical Linear Algebra, and Sections 6.6 and 6.7 in Saad, Iterative Methods for Sparse Linear Systems.
See also 50th Anniversary of Seminal NBS Publication Celebrated (March 2002) and Methods of Conjugate Gradients for Solving Linear Systems (by O'Leary, Natl. Inst. Stand Technol., Spec. Publ. 958).
Matlab: cg_driver.m, mycg.m.

Lecture 7
Thurs., Feb 6:
The Conjugate Gradient method.
Reading: Sections 3.2, 3.3 in the CAAM 454/554 Notes.
Additional Reading: Section 5.1 in Nocedal and Wright, Numerical Optimization, Lecture 38 in Trefethen and Bau, Numerical Linear Algebra, and Sections 6.6 and 6.7 in Saad, Iterative Methods for Sparse Linear Systems.
See also 50th Anniversary of Seminal NBS Publication Celebrated (March 2002) and Methods of Conjugate Gradients for Solving Linear Systems (by O'Leary, Natl. Inst. Stand Technol., Spec. Publ. 958).
Matlab: cg_driver.m, mycg.m.

Lecture 8
Fri., Feb 7
DH1042
:
Krylov subspace methods; orthonormal bases of Krylov subspaces; GMRES.
Reading: Sections 3.4-3.6 in the CAAM 454/554 Notes.
Additional Reading: Lectures 32 in Trefethen and Bau, Numerical Linear Algebra, and Sections 5.1, 5.2, 6.1, 6.2 in Saad, Iterative Methods for Sparse Linear Systems. Matlab: gmres_driver.m, mygmres_simple.m, mygmres.m.

Lecture 9
Tues., Feb 11:
GMRES, CG; convergence of Krylov subspace methods
Reading: Sections 3.6-3.8 in the CAAM 454/554 Notes.
Additional Reading: Lectures 32 in Trefethen and Bau, Numerical Linear Algebra, and Section 6.5 in Saad, Iterative Methods for Sparse Linear Systems.
Matlab: gmres_driver.m, mygmres_simple.m, mygmres.m.

Thurs., Feb 13: no class - Rice Spring recess
Thurs., Feb 18: no class
Thurs., Feb 20: no class
Lecture 10
Tues., Feb. 25:
Convergence of Krylov subspace methods
Reading: Section 3.8 in the CAAM 454/554 Notes.
Matlab: cg_driver_2DLaplace.m, gmres_driver_2Dadvdiff.m.

Lecture 11
Thurs., Feb. 27:
Convergence of Krylov subspace methods.
Reading: Sections 3.8, 3.9.1, 3.9.2 in the CAAM 454/554 Notes.
Matlab: cg_driver_2DLaplace.m, gmres_driver_2Dadvdiff.m.

Lecture 12
Fri., Feb 28
DH1042
::
Preconditioning.
Reading: Sections 3.9.1, 3.9.2 in the CAAM 454/554 Notes.
Matlab: cg_driver_2DLaplace.m, gmres_driver_2Dadvdiff.m.

Lecture 13
Tues., March 3:
Introduction to unconstrained optimization; optimality conditions; convex functions; q-/r-convergence of sequences.
Reading: Chapter 4 in the CAAM 454/554 Notes.
Additional Reading: Chapters 1 and 2 in Nocedal and Wright, Numerical Optimization.

Lecture 14
Thurs., March 5:
Q-/r-convergence of sequences; Newton's method.
Reading: Chapter 5 in the CAAM 454/554 Notes.
Additional Reading: