This is gsl-ref.info, produced by makeinfo version 4.2 from
gsl-ref.texi.

INFO-DIR-SECTION Scientific software
START-INFO-DIR-ENTRY
* gsl-ref: (gsl-ref).                   GNU Scientific Library - Reference
END-INFO-DIR-ENTRY


File: gsl-ref.info,  Node: Root Finding Examples,  Next: Root Finding References and Further Reading,  Prev: Root Finding Algorithms using Derivatives,  Up: One dimensional Root-Finding

Examples
========

   For any root finding algorithm we need to prepare the function to be
solved.  For this example we will use the general quadratic equation
described earlier.  We first need a header file (`demo_fn.h') to define
the function parameters,

     struct quadratic_params
       {
         double a, b, c;
       };
     
     double quadratic (double x, void *params);
     double quadratic_deriv (double x, void *params);
     void quadratic_fdf (double x, void *params,
                         double *y, double *dy);

We place the function definitions in a separate file (`demo_fn.c'),

     double
     quadratic (double x, void *params)
     {
       struct quadratic_params *p
         = (struct quadratic_params *) params;
     
       double a = p->a;
       double b = p->b;
       double c = p->c;
     
       return (a * x + b) * x + c;
     }
     
     double
     quadratic_deriv (double x, void *params)
     {
       struct quadratic_params *p
         = (struct quadratic_params *) params;
     
       double a = p->a;
       double b = p->b;
       double c = p->c;
     
       return 2.0 * a * x + b;
     }
     
     void
     quadratic_fdf (double x, void *params,
                    double *y, double *dy)
     {
       struct quadratic_params *p
         = (struct quadratic_params *) params;
     
       double a = p->a;
       double b = p->b;
       double c = p->c;
     
       *y = (a * x + b) * x + c;
       *dy = 2.0 * a * x + b;
     }

The first program uses the function solver `gsl_root_fsolver_brent' for
Brent's method and the general quadratic defined above to solve the
following equation,

     x^2 - 5 = 0

with solution x = \sqrt 5 = 2.236068...

     #include <stdio.h>
     #include <gsl/gsl_errno.h>
     #include <gsl/gsl_math.h>
     #include <gsl/gsl_roots.h>
     
     #include "demo_fn.h"
     #include "demo_fn.c"
     
     int
     main (void)
     {
       int status;
       int iter = 0, max_iter = 100;
       const gsl_root_fsolver_type *T;
       gsl_root_fsolver *s;
       double r = 0, r_expected = sqrt (5.0);
       double x_lo = 0.0, x_hi = 5.0;
       gsl_function F;
       struct quadratic_params params = {1.0, 0.0, -5.0};
     
       F.function = &quadratic;
       F.params = &params;
     
       T = gsl_root_fsolver_brent;
       s = gsl_root_fsolver_alloc (T);
       gsl_root_fsolver_set (s, &F, x_lo, x_hi);
     
       printf ("using %s method\n",
               gsl_root_fsolver_name (s));
     
       printf ("%5s [%9s, %9s] %9s %10s %9s\n",
               "iter", "lower", "upper", "root",
               "err", "err(est)");
     
       do
         {
           iter++;
           status = gsl_root_fsolver_iterate (s);
           r = gsl_root_fsolver_root (s);
           x_lo = gsl_root_fsolver_x_lower (s);
           x_hi = gsl_root_fsolver_x_upper (s);
           status = gsl_root_test_interval (x_lo, x_hi,
                                            0, 0.001);
     
           if (status == GSL_SUCCESS)
             printf ("Converged:\n");
     
           printf ("%5d [%.7f, %.7f] %.7f %+.7f %.7f\n",
                   iter, x_lo, x_hi,
                   r, r - r_expected,
                   x_hi - x_lo);
         }
       while (status == GSL_CONTINUE && iter < max_iter);
       return status;
     }

Here are the results of the iterations,

     bash$ ./a.out
     using brent method
      iter [    lower,     upper]      root        err  err(est)
         1 [1.0000000, 5.0000000] 1.0000000 -1.2360680 4.0000000
         2 [1.0000000, 3.0000000] 3.0000000 +0.7639320 2.0000000
         3 [2.0000000, 3.0000000] 2.0000000 -0.2360680 1.0000000
         4 [2.2000000, 3.0000000] 2.2000000 -0.0360680 0.8000000
         5 [2.2000000, 2.2366300] 2.2366300 +0.0005621 0.0366300
     Converged:
         6 [2.2360634, 2.2366300] 2.2360634 -0.0000046 0.0005666

If the program is modified to use the bisection solver instead of
Brent's method, by changing `gsl_root_fsolver_brent' to
`gsl_root_fsolver_bisection' the slower convergence of the Bisection
method can be observed,

     bash$ ./a.out
     using bisection method
      iter [    lower,     upper]      root        err  err(est)
         1 [0.0000000, 2.5000000] 1.2500000 -0.9860680 2.5000000
         2 [1.2500000, 2.5000000] 1.8750000 -0.3610680 1.2500000
         3 [1.8750000, 2.5000000] 2.1875000 -0.0485680 0.6250000
         4 [2.1875000, 2.5000000] 2.3437500 +0.1076820 0.3125000
         5 [2.1875000, 2.3437500] 2.2656250 +0.0295570 0.1562500
         6 [2.1875000, 2.2656250] 2.2265625 -0.0095055 0.0781250
         7 [2.2265625, 2.2656250] 2.2460938 +0.0100258 0.0390625
         8 [2.2265625, 2.2460938] 2.2363281 +0.0002601 0.0195312
         9 [2.2265625, 2.2363281] 2.2314453 -0.0046227 0.0097656
        10 [2.2314453, 2.2363281] 2.2338867 -0.0021813 0.0048828
        11 [2.2338867, 2.2363281] 2.2351074 -0.0009606 0.0024414
     Converged:
        12 [2.2351074, 2.2363281] 2.2357178 -0.0003502 0.0012207

   The next program solves the same function using a derivative solver
instead.

     #include <stdio.h>
     #include <gsl/gsl_errno.h>
     #include <gsl/gsl_math.h>
     #include <gsl/gsl_roots.h>
     
     #include "demo_fn.h"
     #include "demo_fn.c"
     
     int
     main (void)
     {
       int status;
       int iter = 0, max_iter = 100;
       const gsl_root_fdfsolver_type *T;
       gsl_root_fdfsolver *s;
       double x0, x = 5.0, r_expected = sqrt (5.0);
       gsl_function_fdf FDF;
       struct quadratic_params params = {1.0, 0.0, -5.0};
     
       FDF.f = &quadratic;
       FDF.df = &quadratic_deriv;
       FDF.fdf = &quadratic_fdf;
       FDF.params = &params;
     
       T = gsl_root_fdfsolver_newton;
       s = gsl_root_fdfsolver_alloc (T);
       gsl_root_fdfsolver_set (s, &FDF, x);
     
       printf ("using %s method\n",
               gsl_root_fdfsolver_name (s));
     
       printf ("%-5s %10s %10s %10s\n",
               "iter", "root", "err", "err(est)");
       do
         {
           iter++;
           status = gsl_root_fdfsolver_iterate (s);
           x0 = x;
           x = gsl_root_fdfsolver_root (s);
           status = gsl_root_test_delta (x, x0, 0, 1e-3);
     
           if (status == GSL_SUCCESS)
             printf ("Converged:\n");
     
           printf ("%5d %10.7f %+10.7f %10.7f\n",
                   iter, x, x - r_expected, x - x0);
         }
       while (status == GSL_CONTINUE && iter < max_iter);
       return status;
     }

Here are the results for Newton's method,

     bash$ ./a.out
     using newton method
     iter        root        err   err(est)
         1  3.0000000 +0.7639320 -2.0000000
         2  2.3333333 +0.0972654 -0.6666667
         3  2.2380952 +0.0020273 -0.0952381
     Converged:
         4  2.2360689 +0.0000009 -0.0020263

Note that the error can be estimated more accurately by taking the
difference between the current iterate and next iterate rather than the
previous iterate.  The other derivative solvers can be investigated by
changing `gsl_root_fdfsolver_newton' to `gsl_root_fdfsolver_secant' or
`gsl_root_fdfsolver_steffenson'.


File: gsl-ref.info,  Node: Root Finding References and Further Reading,  Prev: Root Finding Examples,  Up: One dimensional Root-Finding

References and Further Reading
==============================

For information on the Brent-Dekker algorithm see the following two
papers,

     R. P. Brent, "An algorithm with guaranteed convergence for finding
     a zero of a function", `Computer Journal', 14 (1971) 422-425

     J. C. P. Bus and T. J. Dekker, "Two Efficient Algorithms with
     Guaranteed Convergence for Finding a Zero of a Function", `ACM
     Transactions of Mathematical Software', Vol. 1 No. 4 (1975) 330-345


File: gsl-ref.info,  Node: One dimensional Minimization,  Next: Multidimensional Root-Finding,  Prev: One dimensional Root-Finding,  Up: Top

One dimensional Minimization
****************************

   This chapter describes routines for finding minima of arbitrary
one-dimensional functions.  The library provides low level components
for a variety of iterative minimizers and convergence tests.  These can
be combined by the user to achieve the desired solution, with full
access to the intermediate steps of the algorithms.  Each class of
methods uses the same framework, so that you can switch between
minimizers at runtime without needing to recompile your program.  Each
instance of a minimizer keeps track of its own state, allowing the
minimizers to be used in multi-threaded programs.

   The header file `gsl_min.h' contains prototypes for the minimization
functions and related declarations.  To use the minimization algorithms
to find the maximum of a function simply invert its sign.

* Menu:

* Minimization Overview::
* Minimization Caveats::
* Initializing the Minimizer::
* Providing the function to minimize::
* Minimization Iteration::
* Minimization Stopping Parameters::
* Minimization Algorithms::
* Minimization Examples::
* Minimization References and Further Reading::


File: gsl-ref.info,  Node: Minimization Overview,  Next: Minimization Caveats,  Up: One dimensional Minimization

Overview
========

   The minimization algorithms begin with a bounded region known to
contain a minimum.  The region is described by an lower bound a and an
upper bound b, with an estimate of the location of the minimum x.

The value of the function at x must be less than the value of the
function at the ends of the interval,

     f(a) > f(x) < f(b)

This condition guarantees that a minimum is contained somewhere within
the interval.  On each iteration a new point x' is selected using one
of the available algorithms.  If the new point is a better estimate of
the minimum, f(x') < f(x), then the current estimate of the minimum x
is updated.  The new point also allows the size of the bounded interval
to be reduced, by choosing the most compact set of points which
satisfies the constraint f(a) > f(x) < f(b).  The interval is reduced
until it encloses the true minimum to a desired tolerance.  This
provides a best estimate of the location of the minimum and a rigorous
error estimate.

   Several bracketing algorithms are available within a single
framework.  The user provides a high-level driver for the algorithm,
and the library provides the individual functions necessary for each of
the steps.  There are three main phases of the iteration.  The steps
are,

   * initialize minimizer state, S, for algorithm T

   * update S using the iteration T

   * test S for convergence, and repeat iteration if necessary

The state for the minimizers is held in a `gsl_min_fminimizer' struct.
The updating procedure uses only function evaluations (not derivatives).


File: gsl-ref.info,  Node: Minimization Caveats,  Next: Initializing the Minimizer,  Prev: Minimization Overview,  Up: One dimensional Minimization

Caveats
=======

   Note that minimization functions can only search for one minimum at a
time.  When there are several minima in the search area, the first
minimum to be found will be returned; however it is difficult to predict
which of the minima this will be. _In most cases, no error will be
reported if you try to find a minimum in an area where there is more
than one._

   With all minimization algorithms it can be difficult to determine the
location of the minimum to full numerical precision.  The behavior of
the function in the region of the minimum x^* can be approximated by a
Taylor expansion,

     y = f(x^*) + (1/2) f''(x^*) (x - x^*)^2

and the second term of this expansion can be lost when added to the
first term at finite precision.  This magnifies the error in locating
x^*, making it proportional to \sqrt \epsilon (where \epsilon is the
relative accuracy of the floating point numbers).  For functions with
higher order minima, such as x^4, the magnification of the error is
correspondingly worse.  The best that can be achieved is to converge to
the limit of numerical accuracy in the function values, rather than the
location of the minimum itself.


File: gsl-ref.info,  Node: Initializing the Minimizer,  Next: Providing the function to minimize,  Prev: Minimization Caveats,  Up: One dimensional Minimization

Initializing the Minimizer
==========================

 - Function: gsl_min_fminimizer * gsl_min_fminimizer_alloc (const
          gsl_min_fminimizer_type * T)
     This function returns a pointer to a newly allocated instance of a
     minimizer of type T.  For example, the following code creates an
     instance of a golden section minimizer,

          const gsl_min_fminimizer_type * T
            = gsl_min_fminimizer_goldensection;
          gsl_min_fminimizer * s
            = gsl_min_fminimizer_alloc (T);

     If there is insufficient memory to create the minimizer then the
     function returns a null pointer and the error handler is invoked
     with an error code of `GSL_ENOMEM'.

 - Function: int gsl_min_fminimizer_set (gsl_min_fminimizer * S,
          gsl_function * F, double X_MINIMUM, double X_LOWER, double
          X_UPPER)
     This function sets, or resets, an existing minimizer S to use the
     function F and the initial search interval [X_LOWER, X_UPPER],
     with a guess for the location of the minimum X_MINIMUM.

     If the interval given does not contain a minimum, then the function
     returns an error code of `GSL_FAILURE'.

 - Function: int gsl_min_fminimizer_set_with_values (gsl_min_fminimizer
          * S, gsl_function * F, double X_MINIMUM, double F_MINIMUM,
          double X_LOWER, double F_LOWER, double X_UPPER, double
          F_UPPER)
     This function is equivalent to `gsl_min_fminimizer_set' but uses
     the values F_MINIMUM, F_LOWER and F_UPPER instead of computing
     `f(x_minimum)', `f(x_lower)' and `f(x_upper)'.

 - Function: void gsl_min_fminimizer_free (gsl_min_fminimizer * S)
     This function frees all the memory associated with the minimizer S.

 - Function: const char * gsl_min_fminimizer_name (const
          gsl_min_fminimizer * S)
     This function returns a pointer to the name of the minimizer.  For
     example,

          printf("s is a '%s' minimizer\n",
                 gsl_min_fminimizer_name (s));

     would print something like `s is a 'brent' minimizer'.


File: gsl-ref.info,  Node: Providing the function to minimize,  Next: Minimization Iteration,  Prev: Initializing the Minimizer,  Up: One dimensional Minimization

Providing the function to minimize
==================================

   You must provide a continuous function of one variable for the
minimizers to operate on.  In order to allow for general parameters the
functions are defined by a `gsl_function' data type (*note Providing
the function to solve::).


File: gsl-ref.info,  Node: Minimization Iteration,  Next: Minimization Stopping Parameters,  Prev: Providing the function to minimize,  Up: One dimensional Minimization

Iteration
=========

   The following functions drive the iteration of each algorithm.  Each
function performs one iteration to update the state of any minimizer of
the corresponding type.  The same functions work for all minimizers so
that different methods can be substituted at runtime without
modifications to the code.

 - Function: int gsl_min_fminimizer_iterate (gsl_min_fminimizer * S)
     This function performs a single iteration of the minimizer S.  If
     the iteration encounters an unexpected problem then an error code
     will be returned,

    `GSL_EBADFUNC'
          the iteration encountered a singular point where the function
          evaluated to `Inf' or `NaN'.

    `GSL_FAILURE'
          the algorithm could not improve the current best
          approximation or bounding interval.

   The minimizer maintains a current best estimate of the position of
the minimum at all times, and the current interval bounding the minimum.
This information can be accessed with the following auxiliary functions,

 - Function: double gsl_min_fminimizer_x_minimum (const
          gsl_min_fminimizer * S)
     This function returns the current estimate of the position of the
     minimum for the minimizer S.

 - Function: double gsl_min_fminimizer_x_upper (const
          gsl_min_fminimizer * S)
 - Function: double gsl_min_fminimizer_x_lower (const
          gsl_min_fminimizer * S)
     These functions return the current upper and lower bound of the
     interval for the minimizer S.

 - Function: double gsl_min_fminimizer_f_minimum (const
          gsl_min_fminimizer *S)
 - Function: double gsl_min_fminimizer_f_upper (const
          gsl_min_fminimizer *S)
 - Function: double gsl_min_fminimizer_f_lower (const
          gsl_min_fminimizer *S)
     These functions return the value of the function at the current
     estimate of the minimum and at the upper and lower bounds of
     interval for the minimizer S.


File: gsl-ref.info,  Node: Minimization Stopping Parameters,  Next: Minimization Algorithms,  Prev: Minimization Iteration,  Up: One dimensional Minimization

Stopping Parameters
===================

   A minimization procedure should stop when one of the following
conditions is true:

   * A minimum has been found to within the user-specified precision.

   * A user-specified maximum number of iterations has been reached.

   * An error has occurred.

The handling of these conditions is under user control.  The function
below allows the user to test the precision of the current result.

 - Function: int gsl_min_test_interval (double X_LOWER, double X_UPPER,
          double EPSABS, double EPSREL)
     This function tests for the convergence of the interval [X_LOWER,
     X_UPPER] with absolute error EPSABS and relative error EPSREL.
     The test returns `GSL_SUCCESS' if the following condition is
     achieved,

          |a - b| < epsabs + epsrel min(|a|,|b|)

     when the interval x = [a,b] does not include the origin.  If the
     interval includes the origin then \min(|a|,|b|) is replaced by
     zero (which is the minimum value of |x| over the interval).  This
     ensures that the relative error is accurately estimated for minima
     close to the origin.

     This condition on the interval also implies that any estimate of
     the minimum x_m in the interval satisfies the same condition with
     respect to the true minimum x_m^*,

          |x_m - x_m^*| < epsabs + epsrel x_m^*

     assuming that the true minimum x_m^* is contained within the
     interval.


File: gsl-ref.info,  Node: Minimization Algorithms,  Next: Minimization Examples,  Prev: Minimization Stopping Parameters,  Up: One dimensional Minimization

Minimization Algorithms
=======================

   The minimization algorithms described in this section require an
initial interval which is guaranteed to contain a minimum -- if a and b
are the endpoints of the interval and x is an estimate of the minimum
then f(a) > f(x) < f(b).  This ensures that the function has at least
one minimum somewhere in the interval.  If a valid initial interval is
used then these algorithm cannot fail, provided the function is
well-behaved.

 - Minimizer: gsl_min_fminimizer_goldensection
     The "golden section algorithm" is the simplest method of bracketing
     the minimum of a function.  It is the slowest algorithm provided
     by the library, with linear convergence.

     On each iteration, the algorithm first compares the subintervals
     from the endpoints to the current minimum.  The larger subinterval
     is divided in a golden section (using the famous ratio (3-\sqrt
     5)/2 = 0.3189660...) and the value of the function at this new
     point is calculated.  The new value is used with the constraint
     f(a') > f(x') < f(b') to a select new interval containing the
     minimum, by discarding the least useful point.  This procedure can
     be continued indefinitely until the interval is sufficiently
     small.  Choosing the golden section as the bisection ratio can be
     shown to provide the fastest convergence for this type of
     algorithm.


 - Minimizer: gsl_min_fminimizer_brent
     The "Brent minimization algorithm" combines a parabolic
     interpolation with the golden section algorithm.  This produces a
     fast algorithm which is still robust.

     The outline of the algorithm can be summarized as follows: on each
     iteration Brent's method approximates the function using an
     interpolating parabola through three existing points.  The minimum
     of the parabola is taken as a guess for the minimum.  If it lies
     within the bounds of the current interval then the interpolating
     point is accepted, and used to generate a smaller interval.  If
     the interpolating point is not accepted then the algorithm falls
     back to an ordinary golden section step.  The full details of
     Brent's method include some additional checks to improve
     convergence.


File: gsl-ref.info,  Node: Minimization Examples,  Next: Minimization References and Further Reading,  Prev: Minimization Algorithms,  Up: One dimensional Minimization

Examples
========

   The following program uses the Brent algorithm to find the minimum of
the function f(x) = \cos(x) + 1, which occurs at x = \pi.  The starting
interval is (0,6), with an initial guess for the minimum of 2.

     #include <stdio.h>
     #include <gsl/gsl_errno.h>
     #include <gsl/gsl_math.h>
     #include <gsl/gsl_min.h>
     
     double fn1 (double x, void * params)
     {
       return cos(x) + 1.0;
     }
     
     int
     main (void)
     {
       int status;
       int iter = 0, max_iter = 100;
       const gsl_min_fminimizer_type *T;
       gsl_min_fminimizer *s;
       double m = 2.0, m_expected = M_PI;
       double a = 0.0, b = 6.0;
       gsl_function F;
     
       F.function = &fn1;
       F.params = 0;
     
       T = gsl_min_fminimizer_brent;
       s = gsl_min_fminimizer_alloc (T);
       gsl_min_fminimizer_set (s, &F, m, a, b);
     
       printf ("using %s method\n",
               gsl_min_fminimizer_name (s));
     
       printf ("%5s [%9s, %9s] %9s %10s %9s\n",
               "iter", "lower", "upper", "min",
               "err", "err(est)");
     
       printf ("%5d [%.7f, %.7f] %.7f %+.7f %.7f\n",
               iter, a, b,
               m, m - m_expected, b - a);
     
       do
         {
           iter++;
           status = gsl_min_fminimizer_iterate (s);
     
           m = gsl_min_fminimizer_x_minimum (s);
           a = gsl_min_fminimizer_x_lower (s);
           b = gsl_min_fminimizer_x_upper (s);
     
           status
             = gsl_min_test_interval (a, b, 0.001, 0.0);
     
           if (status == GSL_SUCCESS)
             printf ("Converged:\n");
     
           printf ("%5d [%.7f, %.7f] "
                   "%.7f %.7f %+.7f %.7f\n",
                   iter, a, b,
                   m, m_expected, m - m_expected, b - a);
         }
       while (status == GSL_CONTINUE && iter < max_iter);
     
       return status;
     }

Here are the results of the minimization procedure.

     bash$ ./a.out
         0 [0.0000000, 6.0000000] 2.0000000 -1.1415927 6.0000000
         1 [2.0000000, 6.0000000] 3.2758640 +0.1342713 4.0000000
         2 [2.0000000, 3.2831929] 3.2758640 +0.1342713 1.2831929
         3 [2.8689068, 3.2831929] 3.2758640 +0.1342713 0.4142862
         4 [2.8689068, 3.2831929] 3.2758640 +0.1342713 0.4142862
         5 [2.8689068, 3.2758640] 3.1460585 +0.0044658 0.4069572
         6 [3.1346075, 3.2758640] 3.1460585 +0.0044658 0.1412565
         7 [3.1346075, 3.1874620] 3.1460585 +0.0044658 0.0528545
         8 [3.1346075, 3.1460585] 3.1460585 +0.0044658 0.0114510
         9 [3.1346075, 3.1460585] 3.1424060 +0.0008133 0.0114510
        10 [3.1346075, 3.1424060] 3.1415885 -0.0000041 0.0077985
     Converged:
        11 [3.1415885, 3.1424060] 3.1415927 -0.0000000 0.0008175


File: gsl-ref.info,  Node: Minimization References and Further Reading,  Prev: Minimization Examples,  Up: One dimensional Minimization

References and Further Reading
==============================

Further information on Brent's algorithm is available in the following
book,

     Richard Brent, `Algorithms for minimization without derivatives',
     Prentice-Hall (1973), republished by Dover in paperback (2002),
     ISBN 0-486-41998-3.


File: gsl-ref.info,  Node: Multidimensional Root-Finding,  Next: Multidimensional Minimization,  Prev: One dimensional Minimization,  Up: Top

Multidimensional Root-Finding
*****************************

   This chapter describes functions for multidimensional root-finding
(solving nonlinear systems with n equations in n unknowns).  The
library provides low level components for a variety of iterative
solvers and convergence tests.  These can be combined by the user to
achieve the desired solution, with full access to the intermediate
steps of the iteration.  Each class of methods uses the same framework,
so that you can switch between solvers at runtime without needing to
recompile your program.  Each instance of a solver keeps track of its
own state, allowing the solvers to be used in multi-threaded programs.
The solvers are based on the original Fortran library MINPACK.

   The header file `gsl_multiroots.h' contains prototypes for the
multidimensional root finding functions and related declarations.

* Menu:

* Overview of Multidimensional Root Finding::
* Initializing the Multidimensional Solver::
* Providing the multidimensional system of equations to solve::
* Iteration of the multidimensional solver::
* Search Stopping Parameters for the multidimensional solver::
* Algorithms using Derivatives::
* Algorithms without Derivatives::
* Example programs for Multidimensional Root finding::
* References and Further Reading for Multidimensional Root Finding::


File: gsl-ref.info,  Node: Overview of Multidimensional Root Finding,  Next: Initializing the Multidimensional Solver,  Up: Multidimensional Root-Finding

Overview
========

   The problem of multidimensional root finding requires the
simultaneous solution of n equations, f_i, in n variables, x_i,

     f_i (x_1, ..., x_n) = 0    for i = 1 ... n.

In general there are no bracketing methods available for n dimensional
systems, and no way of knowing whether any solutions exist.  All
algorithms proceed from an initial guess using a variant of the Newton
iteration,

     x -> x' = x - J^{-1} f(x)

where x, f are vector quantities and J is the Jacobian matrix J_{ij} =
d f_i / d x_j.  Additional strategies can be used to enlarge the region
of convergence.  These include requiring a decrease in the norm |f| on
each step proposed by Newton's method, or taking steepest-descent steps
in the direction of the negative gradient of |f|.

   Several root-finding algorithms are available within a single
framework.  The user provides a high-level driver for the algorithms,
and the library provides the individual functions necessary for each of
the steps.  There are three main phases of the iteration.  The steps
are,

   * initialize solver state, S, for algorithm T

   * update S using the iteration T

   * test S for convergence, and repeat iteration if necessary

The evaluation of the Jacobian matrix can be problematic, either because
programming the derivatives is intractable or because computation of the
n^2 terms of the matrix becomes too expensive.  For these reasons the
algorithms provided by the library are divided into two classes
according to whether the derivatives are available or not.

   The state for solvers with an analytic Jacobian matrix is held in a
`gsl_multiroot_fdfsolver' struct.  The updating procedure requires both
the function and its derivatives to be supplied by the user.

   The state for solvers which do not use an analytic Jacobian matrix is
held in a `gsl_multiroot_fsolver' struct.  The updating procedure uses
only function evaluations (not derivatives).  The algorithms estimate
the matrix J or J^{-1} by approximate methods.


File: gsl-ref.info,  Node: Initializing the Multidimensional Solver,  Next: Providing the multidimensional system of equations to solve,  Prev: Overview of Multidimensional Root Finding,  Up: Multidimensional Root-Finding

Initializing the Solver
=======================

   The following functions initialize a multidimensional solver, either
with or without derivatives.  The solver itself depends only on the
dimension of the problem and the algorithm and can be reused for
different problems.

 - Function: gsl_multiroot_fsolver * gsl_multiroot_fsolver_alloc (const
          gsl_multiroot_fsolver_type * T, size_t N)
     This function returns a pointer to a newly allocated instance of a
     solver of type T for a system of N dimensions.  For example, the
     following code creates an instance of a hybrid solver, to solve a
     3-dimensional system of equations.

          const gsl_multiroot_fsolver_type * T
              = gsl_multiroot_fsolver_hybrid;
          gsl_multiroot_fsolver * s
              = gsl_multiroot_fsolver_alloc (T, 3);

     If there is insufficient memory to create the solver then the
     function returns a null pointer and the error handler is invoked
     with an error code of `GSL_ENOMEM'.

 - Function: gsl_multiroot_fdfsolver * gsl_multiroot_fdfsolver_alloc
          (const gsl_multiroot_fdfsolver_type * T, size_t N)
     This function returns a pointer to a newly allocated instance of a
     derivative solver of type T for a system of N dimensions.  For
     example, the following code creates an instance of a
     Newton-Raphson solver, for a 2-dimensional system of equations.

          const gsl_multiroot_fdfsolver_type * T
              = gsl_multiroot_fdfsolver_newton;
          gsl_multiroot_fdfsolver * s =
              gsl_multiroot_fdfsolver_alloc (T, 2);

     If there is insufficient memory to create the solver then the
     function returns a null pointer and the error handler is invoked
     with an error code of `GSL_ENOMEM'.

 - Function: int gsl_multiroot_fsolver_set (gsl_multiroot_fsolver * S,
          gsl_multiroot_function * F, gsl_vector * X)
     This function sets, or resets, an existing solver S to use the
     function F and the initial guess X.

 - Function: int gsl_multiroot_fdfsolver_set (gsl_multiroot_fdfsolver *
          S, gsl_function_fdf * FDF, gsl_vector * X)
     This function sets, or resets, an existing solver S to use the
     function and derivative FDF and the initial guess X.

 - Function: void gsl_multiroot_fsolver_free (gsl_multiroot_fsolver * S)
 - Function: void gsl_multiroot_fdfsolver_free (gsl_multiroot_fdfsolver
          * S)
     These functions free all the memory associated with the solver S.

 - Function: const char * gsl_multiroot_fsolver_name (const
          gsl_multiroot_fsolver * S)
 - Function: const char * gsl_multiroot_fdfsolver_name (const
          gsl_multiroot_fdfsolver * S)
     These functions return a pointer to the name of the solver.  For
     example,

          printf("s is a '%s' solver\n",
                 gsl_multiroot_fdfsolver_name (s));

     would print something like `s is a 'newton' solver'.


File: gsl-ref.info,  Node: Providing the multidimensional system of equations to solve,  Next: Iteration of the multidimensional solver,  Prev: Initializing the Multidimensional Solver,  Up: Multidimensional Root-Finding

Providing the function to solve
===============================

   You must provide n functions of n variables for the root finders to
operate on.  In order to allow for general parameters the functions are
defined by the following data types:

 - Data Type: gsl_multiroot_function
     This data type defines a general system of functions with
     parameters.

    `int (* f) (const gsl_vector * X, void * PARAMS, gsl_vector * F)'
          this function should store the vector result f(x,params) in F
          for argument X and parameters PARAMS, returning an
          appropriate error code if the function cannot be computed.

    `size_t N'
          the dimension of the system, i.e. the number of components of
          the vectors X and F.

    `void * PARAMS'
          a pointer to the parameters of the function.

Here is an example using Powell's test function,

     f_1(x) = A x_0 x_1 - 1,
     f_2(x) = exp(-x_0) + exp(-x_1) - (1 + 1/A)

with A = 10^4.  The following code defines a `gsl_multiroot_function'
system `F' which you could pass to a solver:

     struct powell_params { double A; };
     
     int
     powell (gsl_vector * x, void * p, gsl_vector * f) {
        struct powell_params * params
          = *(struct powell_params *)p;
        double A = (params->A);
        double x0 = gsl_vector_get(x,0);
        double x1 = gsl_vector_get(x,1);
     
        gsl_vector_set (f, 0, A * x0 * x1 - 1)
        gsl_vector_set (f, 1, (exp(-x0) + exp(-x1)
                               - (1.0 + 1.0/A)))
        return GSL_SUCCESS
     }
     
     gsl_multiroot_function F;
     struct powell_params params = { 10000.0 };
     
     F.function = &powell;
     F.n = 2;
     F.params = &params;

 - Data Type: gsl_multiroot_function_fdf
     This data type defines a general system of functions with
     parameters and the corresponding Jacobian matrix of derivatives,

    `int (* f) (const gsl_vector * X, void * PARAMS, gsl_vector * F)'
          this function should store the vector result f(x,params) in F
          for argument X and parameters PARAMS, returning an
          appropriate error code if the function cannot be computed.

    `int (* df) (const gsl_vector * X, void * PARAMS, gsl_matrix * J)'
          this function should store the N-by-N matrix result J_ij = d
          f_i(x,params) / d x_j in J for argument X and parameters
          PARAMS, returning an appropriate error code if the function
          cannot be computed.

    `int (* fdf) (const gsl_vector * X, void * PARAMS, gsl_vector * F, gsl_matrix * J)'
          This function should set the values of the F and J as above,
          for arguments X and parameters PARAMS.  This function provides
          an optimization of the separate functions for f(x) and J(x) -
          it is always faster to compute the function and its
          derivative at the same time.

    `size_t N'
          the dimension of the system, i.e. the number of components of
          the vectors X and F.

    `void * PARAMS'
          a pointer to the parameters of the function.

The example of Powell's test function defined above can be extended to
include analytic derivatives using the following code,

     int
     powell_df (gsl_vector * x, void * p, gsl_matrix * J)
     {
        struct powell_params * params
          = *(struct powell_params *)p;
        double A = (params->A);
        double x0 = gsl_vector_get(x,0);
        double x1 = gsl_vector_get(x,1);
        gsl_matrix_set (J, 0, 0, A * x1)
        gsl_matrix_set (J, 0, 1, A * x0)
        gsl_matrix_set (J, 1, 0, -exp(-x0))
        gsl_matrix_set (J, 1, 1, -exp(-x1))
        return GSL_SUCCESS
     }
     
     int
     powell_fdf (gsl_vector * x, void * p,
                 gsl_matrix * f, gsl_matrix * J) {
        struct powell_params * params
          = *(struct powell_params *)p;
        double A = (params->A);
        double x0 = gsl_vector_get(x,0);
        double x1 = gsl_vector_get(x,1);
     
        double u0 = exp(-x0);
        double u1 = exp(-x1);
     
        gsl_vector_set (f, 0, A * x0 * x1 - 1)
        gsl_vector_set (f, 1, u0 + u1 - (1 + 1/A))
     
        gsl_matrix_set (J, 0, 0, A * x1)
        gsl_matrix_set (J, 0, 1, A * x0)
        gsl_matrix_set (J, 1, 0, -u0)
        gsl_matrix_set (J, 1, 1, -u1)
        return GSL_SUCCESS
     }
     
     gsl_multiroot_function_fdf FDF;
     
     FDF.f = &powell_f;
     FDF.df = &powell_df;
     FDF.fdf = &powell_fdf;
     FDF.n = 2;
     FDF.params = 0;

Note that the function `powell_fdf' is able to reuse existing terms
from the function when calculating the Jacobian, thus saving time.


File: gsl-ref.info,  Node: Iteration of the multidimensional solver,  Next: Search Stopping Parameters for the multidimensional solver,  Prev: Providing the multidimensional system of equations to solve,  Up: Multidimensional Root-Finding

Iteration
=========

   The following functions drive the iteration of each algorithm.  Each
function performs one iteration to update the state of any solver of the
corresponding type.  The same functions work for all solvers so that
different methods can be substituted at runtime without modifications to
the code.

 - Function: int gsl_multiroot_fsolver_iterate (gsl_multiroot_fsolver *
          S)
 - Function: int gsl_multiroot_fdfsolver_iterate
          (gsl_multiroot_fdfsolver * S)
     These functions perform a single iteration of the solver S.  If the
     iteration encounters an unexpected problem then an error code will
     be returned,

    `GSL_EBADFUNC'
          the iteration encountered a singular point where the function
          or its derivative evaluated to `Inf' or `NaN'.

    `GSL_ENOPROG'
          the iteration is not making any progress, preventing the
          algorithm from continuing.

   The solver maintains a current best estimate of the root at all
times.  This information can be accessed with the following auxiliary
functions,

 - Function: gsl_vector * gsl_multiroot_fsolver_root (const
          gsl_multiroot_fsolver * S)
 - Function: gsl_vector * gsl_multiroot_fdfsolver_root (const
          gsl_multiroot_fdfsolver * S)
     These functions return the current estimate of the root for the
     solver S.

 - Function: gsl_vector * gsl_multiroot_fsolver_f (const
          gsl_multiroot_fsolver * S)
 - Function: gsl_vector * gsl_multiroot_fdfsolver_f (const
          gsl_multiroot_fdfsolver * S)
     These functions return the function value f(x) at the current
     estimate of the root for the solver S.

 - Function: gsl_vector * gsl_multiroot_fsolver_dx (const
          gsl_multiroot_fsolver * S)
 - Function: gsl_vector * gsl_multiroot_fdfsolver_dx (const
          gsl_multiroot_fdfsolver * S)
     These functions return the last step dx taken by the solver S.


File: gsl-ref.info,  Node: Search Stopping Parameters for the multidimensional solver,  Next: Algorithms using Derivatives,  Prev: Iteration of the multidimensional solver,  Up: Multidimensional Root-Finding

Search Stopping Parameters
==========================

   A root finding procedure should stop when one of the following
conditions is true:

   * A multidimensional root has been found to within the
     user-specified precision.

   * A user-specified maximum number of iterations has been reached.

   * An error has occurred.

The handling of these conditions is under user control.  The functions
below allow the user to test the precision of the current result in
several standard ways.

 - Function: int gsl_multiroot_test_delta (const gsl_vector * DX, const
          gsl_vector * X, double EPSABS, double EPSREL)
     This function tests for the convergence of the sequence by
     comparing the last step DX with the absolute error EPSABS and
     relative error EPSREL to the current position X.  The test returns
     `GSL_SUCCESS' if the following condition is achieved,

          |dx_i| < epsabs + epsrel |x_i|

     for each component of X and returns `GSL_CONTINUE' otherwise.

 - Function: int gsl_multiroot_test_residual (const gsl_vector * F,
          double EPSABS)
     This function tests the residual value F against the absolute
     error bound EPSABS.  The test returns `GSL_SUCCESS' if the
     following condition is achieved,

          \sum_i |f_i| < epsabs

     and returns `GSL_CONTINUE' otherwise.  This criterion is suitable
     for situations where the precise location of the root, x, is
     unimportant provided a value can be found where the residual is
     small enough.


File: gsl-ref.info,  Node: Algorithms using Derivatives,  Next: Algorithms without Derivatives,  Prev: Search Stopping Parameters for the multidimensional solver,  Up: Multidimensional Root-Finding

Algorithms using Derivatives
============================

   The root finding algorithms described in this section make use of
both the function and its derivative.  They require an initial guess
for the location of the root, but there is no absolute guarantee of
convergence - the function must be suitable for this technique and the
initial guess must be sufficiently close to the root for it to work.
When the conditions are satisfied then convergence is quadratic.

 - Derivative Solver: gsl_multiroot_fdfsolver_hybridsj
     This is a modified version of Powell's Hybrid method as
     implemented in the HYBRJ algorithm in MINPACK.  Minpack was
     written by Jorge J. More', Burton S. Garbow and Kenneth E.
     Hillstrom.  The Hybrid algorithm retains the fast convergence of
     Newton's method but will also reduce the residual when Newton's
     method is unreliable.

     The algorithm uses a generalized trust region to keep each step
     under control.  In order to be accepted a proposed new position x'
     must satisfy the condition |D (x' - x)| < \delta, where D is a
     diagonal scaling matrix and \delta is the size of the trust
     region.  The components of D are computed internally, using the
     column norms of the Jacobian to estimate the sensitivity of the
     residual to each component of x.  This improves the behavior of the
     algorithm for badly scaled functions.

     On each iteration the algorithm first determines the standard
     Newton step by solving the system J dx = - f.  If this step falls
     inside the trust region it is used as a trial step in the next
     stage.  If not, the algorithm uses the linear combination of the
     Newton and gradient directions which is predicted to minimize the
     norm of the function while staying inside the trust region.

          dx = - \alpha J^{-1} f(x) - \beta \nabla |f(x)|^2

     This combination of Newton and gradient directions is referred to
     as a "dogleg step".

     The proposed step is now tested by evaluating the function at the
     resulting point, x'.  If the step reduces the norm of the function
     sufficiently then it is accepted and size of the trust region is
     increased.  If the proposed step fails to improve the solution
     then the size of the trust region is decreased and another trial
     step is computed.

     The speed of the algorithm is increased by computing the changes
     to the Jacobian approximately, using a rank-1 update.  If two
     successive attempts fail to reduce the residual then the full
     Jacobian is recomputed.  The algorithm also monitors the progress
     of the solution and returns an error if several steps fail to make
     any improvement,

    `GSL_ENOPROG'
          the iteration is not making any progress, preventing the
          algorithm from continuing.

    `GSL_ENOPROGJ'
          re-evaluations of the Jacobian indicate that the iteration is
          not making any progress, preventing the algorithm from
          continuing.


 - Derivative Solver: gsl_multiroot_fdfsolver_hybridj
     This algorithm is an unscaled version of `hybridsj'.  The steps are
     controlled by a spherical trust region |x' - x| < \delta, instead
     of a generalized region.  This can be useful if the generalized
     region estimated by `hybridsj' is inappropriate.

 - Derivative Solver: gsl_multiroot_fdfsolver_newton
     Newton's Method is the standard root-polishing algorithm.  The
     algorithm begins with an initial guess for the location of the
     solution.  On each iteration a linear approximation to the
     function F is used to estimate the step which will zero all the
     components of the residual.  The iteration is defined by the
     following sequence,

          x -> x' = x - J^{-1} f(x)

     where the Jacobian matrix J is computed from the derivative
     functions provided by F.  The step dx is obtained by solving the
     linear system,

          J dx = - f(x)

     using LU decomposition.

 - Derivative Solver: gsl_multiroot_fdfsolver_gnewton
     This is a modified version of Newton's method which attempts to
     improve global convergence by requiring every step to reduce the
     Euclidean norm of the residual, |f(x)|.  If the Newton step leads
     to an increase in the norm then a reduced step of relative size,

          t = (\sqrt(1 + 6 r) - 1) / (3 r)

     is proposed, with r being the ratio of norms |f(x')|^2/|f(x)|^2.
     This procedure is repeated until a suitable step size is found.


File: gsl-ref.info,  Node: Algorithms without Derivatives,  Next: Example programs for Multidimensional Root finding,  Prev: Algorithms using Derivatives,  Up: Multidimensional Root-Finding

Algorithms without Derivatives
==============================

   The algorithms described in this section do not require any
derivative information to be supplied by the user.  Any derivatives
needed are approximated from by finite difference.

 - Solver: gsl_multiroot_fsolver_hybrids
     This is a version of the Hybrid algorithm which replaces calls to
     the Jacobian function by its finite difference approximation.  The
     finite difference approximation is computed using
     `gsl_multiroots_fdjac' with a relative step size of
     `GSL_SQRT_DBL_EPSILON'.

 - Solver: gsl_multiroot_fsolver_hybrid
     This is a finite difference version of the Hybrid algorithm without
     internal scaling.

 - Solver: gsl_multiroot_fsolver_dnewton
     The "discrete Newton algorithm" is the simplest method of solving a
     multidimensional system.  It uses the Newton iteration

          x -> x - J^{-1} f(x)

     where the Jacobian matrix J is approximated by taking finite
     differences of the function F.  The approximation scheme used by
     this implementation is,

          J_{ij} = (f_i(x + \delta_j) - f_i(x)) /  \delta_j

     where \delta_j is a step of size \sqrt\epsilon |x_j| with \epsilon
     being the machine precision (\epsilon \approx 2.22 \times 10^-16).
     The order of convergence of Newton's algorithm is quadratic, but
     the finite differences require n^2 function evaluations on each
     iteration.  The algorithm may become unstable if the finite
     differences are not a good approximation to the true derivatives.

 - Solver: gsl_multiroot_fsolver_broyden
     The "Broyden algorithm" is a version of the discrete Newton
     algorithm which attempts to avoids the expensive update of the
     Jacobian matrix on each iteration.  The changes to the Jacobian
     are also approximated, using a rank-1 update,

          J^{-1} \to J^{-1} - (J^{-1} df - dx) dx^T J^{-1} / dx^T J^{-1} df

     where the vectors dx and df are the changes in x and f.  On the
     first iteration the inverse Jacobian is estimated using finite
     differences, as in the discrete Newton algorithm.

     This approximation gives a fast update but is unreliable if the
     changes are not small, and the estimate of the inverse Jacobian
     becomes worse as time passes.  The algorithm has a tendency to
     become unstable unless it starts close to the root.  The Jacobian
     is refreshed if this instability is detected (consult the source
     for details).

     This algorithm is not recommended and is included only for
     demonstration purposes.

