A full-Newton approach to separable nonlinear least squares problems and its application to discrete least squares rational approximation

Abstract

We consider a class of non-linear least squares problems that are widely used in fitting experimental data. A defining characteristic of the models we will consider is that the solution parameters may be separated into two classes, those that enter the problem linearly and those that enter non-linearly. Problems of this type are known as separable non-linear least squares (SNLLS) problems and are often solved using a Gauss-Newton algorithm that was developed in Golub and Pereyra [SIAM J. Numer. Anal., 10 (1973), pp. 413–432] and has been very widely applied. We develop a full-Newton algorithm for solving this problem. Exploiting the structure of the general problem leads to a surprisingly compact algorithm which exhibits all of the excellent characteristics of the full-Newton approach (e.g. rapid convergence on problems with large residuals). Moreover, for certain problems of quite general interest, the per iteration cost for the full-Newton algorithm compares quite favorably with that of the Gauss-Newton algorithm. We explore one such problem, that of discrete least-squares fitting of rational functions.