[x,iter] = MyGN(func,x,tol,maxiter,prt,varargin);
where func is a function name that returns [r,J],
prt is a switch for printing iteration info or not,
and varargin is a variable list of input arguments
to be used by function func. The stopping criterion
should be norm(g) < tol, where g = J'*r is the
gradient vector.
Download and unzip the set of codes in nls.zip. Download the instructor's codes ZyGN.p and ZyLM.p. Study and run the test code testNLS.m. Without input, it runs the instructor's codes (CodeRange = 3:6) by default. We included 33 test problems, as described in the paper MGH.pdf. The test problems have already been coded. A program FdNewton.m is included that implements Newton's method for solving nonlinear systems using a finite-difference scheme to approximate Jacobian. You may start your programming by studying and modifying this program.
Submit your code and printout from the test program testNLS.m for the following two runs:
testNLS(1, 1.e-4,1:4); (easy run)
testNLS(10,1.e-8,1:4); (hard run)
function [x,sflag,hist] = myBFGS(func,x0,tol,maxit) % % Quasi-Newton method using inverse BFGS update % (if you do backtracking, make sure you check y'*s > 0) % % Usage: % [x,sflag,hist] = myBFGS(func,x0,tol,maxit); % % INPUT: % func - matlab function for f and g evaluations % x0 - starting point (column vector) % tol - stopping tolerance for gradient norm % maxit - maximum number of iterations % % OUTPUT: % x - computed approximate solution % sflag - status flag, contains number of iterations % if convergence achieved or 0 if maxiter reached % hist(1,:) - vector of function values at all iterations % hist(2,:) - vector of gradient norms at all iterations--- Use the identity matrix for the initial Hessian approximation.
function [x,iter,nf] = myGD(func,x0,tol,maxit,varargin)to minimize a function defined by the function handle func starting from initial guess x0. You interface func by
[f,g] = feval(func,x,varargin{:}); % function and gradient
f = feval(func,x,varargin{:}); % function value only
Use back-tracking line search and choose your own parameters.
The stopping criterion is either
crit := norm(g)/(1 + abs(f)) < tol;or the maximum iteration number maxit is reached. The outputs include the approximate solution, the iteration number taken, and the number of function evaluations taken. You should mimic the printout format as the instructor's code does.
Download the test scripts testDenoise.m and testDeblur.m, the functions TVL2N.m and TVL2B.m, and the instructor's code zGD.p. Run them (you will need image processing toolbox). Submit your code and printout including graphs, and a short report (typed) explaining what you observe in these experiments.
function [x,iter,nf] = myCG(func,x0,tol,maxit,varargin)Test the search direction for descent property at each iteration. If not descent, use the negative gradient instead. Download the test scripts and run them with your codes: testDenoiseCG.m and testDeblurCG.m. Submit your code and printout including graphs, and a short write-up (typed) explaining what you observe in these experiments.
Download the test script testGM1p.m and the instructor's code zGMRES1p.p. You can run the script without your code. When your code is ready, run the test script for n = 300. Submit a copy of your program, the printput (graph not necessary), and a short write-up (typed) explaining what you observe in this experiment.
Download the test scripts testGM2.m and testGM2a.m, and the instructor's code zGMRES2.p. You can run these scripts without your code. When your code is ready, run testGM2 for m = 60 or larger (as large as you see fit), and also run testGM2a. Submit a copy of your program, the printputs (graph included) from your runs, and a short write-up (typed) explaining what you observe in the experiment (testGM2a).
Download the test script testGM1.m, the function fdm4pde1.m for matrix generation, and the instructor's GMRES code zGMRES1.p. You can run the test script with or without your code.
After you have written your code, start running the test script for m = 30, and for 2 larger m values, the larger the better, as long as your code can handle it comfortably on the computer you use. Submit a copy of your program and the printout (including graphs) for your 3 runs.
Download the test script testsor.m and the instructor's code zSOR.p. You can run "testsor.m" with or without your code.
After you have debugged your code, run "testsor.m" for N = 100,200,300 with the best omega values you can get. Submit a copy of your program and the printout for the 3 runs.
(2) Prove that the scheme x <-- x + a*(b-Ax) converges for any initial guess if the matrix A is symmetric positive definite and a>0 is less than 2 divided by the maximum eigenvalue of A.
(3) Prove that GS converges for any initial guess if the matrix A is symmetric positive definite.
to implement the Jacobi and Gauss-Seidel methods, respectively, where maxit is the maximum number of iterations. Use the zero vector to start. The stopping criterion is that the residual 2-norm, norm(b-A*x), becomes less than tol*(1 + norm(b)). The output info is a character string indicating the status of termination: either "Converged at iteration ?" or "Maxit reached". And res is a vector containing the residuals, norm(b-A*x), for all iterations. Count the iterations from 1.
Download the instructor's codes zJacobi07.p, zGS07.p) and the test script test1.m. Read the test script.
Run test1.m for N=40, 60, 80 and for both method = 'Jacobi' and method = 'GS'. Submit your programs and output from these runs.