General Xxxxx-Xxxxxx Approach Sample Clauses

General Xxxxx-Xxxxxx Approach. To simplify notation, we can write the nonlinear least squares problem given in equation (3.2) as min ψ(z) = min ǁf (z)ǁ2 , (3.8) z z 2 where zT = [ xT yT ] and λI 0 f (z) = f (x, y) = Σ A(y) Σ x − Σ b Σ . The nonlinear least squares problem (3.8) can be solved using a Xxxxx- Xxxxxx method [31, 86, 104, 108], which is an iterative algorithm that com- putes zk+1 = zk + dk , k = 0, 1, 2, . . . where dk = −(ψ^jj(zk))−1ψj(zk) , ^ ψ ψ z0 is an initial guess, and ψ jj is an approximation of the Hessian ψjj. It is not difficult to show that ψj = JT f , and an approximation of ψjj is given by ψ^jj = JT Jψ, where Jψ is the Jacobian matrix; for our problem it can be Σ Σ Σ written as Jψ = Σ fx fy = ∂f (x, y) ∂x ∂f (x, y) ∂y . If we define r = −f = b−A(y) x, then the computation to update the search direction dk at each Xxxxx-Xxxxxx iteration is equivalent to solving a least squares problem of the form: d 2 min ǁJψd − rǁ2 . (3.9) To summarize, then, a Xxxxx-Xxxxxx method applied to (3.8) has the basic form: end d zk+1 = zk + dk k ψ J d − r ǁ k d = argmin ǁ for k = 0, 0, 0, . . . xx = b − A(yk) xk Σ y0 0 x Σ choose initial z = General Xxxxx-Xxxxxx Algorithm This general Xxxxx-Xxxxxx approach can work well, but constructing and solving linear systems with Jψ can be very expensive. Effective precondition- ers for (3.9) may be difficult to find. Moreover, it requires either specifying a priori the regularization parameter λ or estimating it within a nonlinear it- erative scheme; see [61] and the references therein. A further difficulty when using a fully coupled approach is that we do not take algorithmic advantage of the fact that the problem is strongly convex in x. Thus, we may take very small steps due to the nonlinearity induced by y. Instead, we would like to adapt the variable projection method for separable nonlinear least squares problems to work for large-scale ill-posed inverse problems and de- velop an approach where the regularization parameter can be estimated by the algorithm.