Nonlinear Regression

XLISP-STAT allows the construction of nonlinear, normal regression models. The present implementation is experimental (even more experimental than the rest of the program). The definitions needed for nonlinear regression are in the file "2D nonlin.lsp on the distribution disk. This file is not loaded automatically at start up; you should load it now, using the Load item on the File menu or the "2D load command, to carry out the calculations in this section.

As an example, suppose we have data sets "2D x and "2D y given by

	(def x (list 2.000 2.000 0.667 0.667 0.400 0.400
	             0.286 0.286 0.222 0.222 0.200 0.200))
	(def y (list 0.0615 0.0527 0.0334 0.0258 0.0138 0.0258
	             0.0129 0.0183 0.0083 0.0169 0.0129 0.0087))
and we would like to fit a normal nonlinear regression model with the Michaelis-Menten mean function

η(x) = $\displaystyle {\frac{{\theta_{1}x}}{{\theta_{2}+x}}}$

to this data. The function "2D f defined by
(defun f (b) (/ (* (select b 0) x) (+ (select b 1) x)))
will compute the list of mean response values at the points in "2D x for a parameter list "2D b. Using these definitions, which are contained in the file "2D mikement.lsp in the "2D Data folder of the distribution disk, we can construct a nonlinear regression model using the "2D nreg-model function: 23
> (def mikement (nreg-model #'f y (list 1 1)))

Least Squares Estimates:

Parameter  0:           0.1056427   (0.01760005)
Parameter  1:             1.70269   (0.4757766)

R Squared:              0.9379065
Sigma hat:             0.004483935
Number of cases:               12
Degrees of freedom:            10

MIKEMENT
>

The function "2D nreg-model requires three arguments, a mean function, a list or vector of observed response values, and an initial guess for the parameter vector. In addition it can take two keyword arguments, "2D :epsilon to specify a convergence criterion and "2D :count-limit, a limit on the number of iterations. By default these values are .001 and 20, respectively. A simple, unmodified Gauss-Newton algorithm based on numerical derivatives is used for fitting the model.

To see how you can analyze the model further you can send mikement the "2D :help message. The result is very similar to the help information for a linear regression model. The reason for this is simple. Nonlinear regression models have been implemented as model objects. The inheritance mechanism in XLISP-STAT allows new objects to be defined as variations on old objects. The new object inherits all the methods defined for the old object. If some of these methods don't apply directly to the new object they can be redefined, and new methods that only make sense for the new object can be added. I have defined "2D nreg-model-proto, the prototype for nonlinear regression models, as an object inheriting from of the linear regression model prototype. The internal data, the method for computing estimates and the method of computing fitted values have been modified, but the other methods remain unchanged. Once the model has been fit the Jacobian of the mean function at the estimated parameter values is used as the X matrix. In terms of this definition most of the methods for linear regression, the methods "2D :coef-standard-errors and "2D :leverages for example, still make sense, at least as first order asymptotic approximations.