
Large-scale ill-posed problems arise in many applications, for instance in image deblurring. The solution of these problems has recently received considerable attention. This talk surveys several kinds of iterative methods for large ill-posed problems, including methods for constrained problems.
[Return to schedule]We consider the web hyperlink matrix used by Google for computing
the PageRank whose form is given by
where
is a row stochastic matrix,
is a row stochastic
rank one
matrix and
.
We determine the analytic expression of
the Jordan form of
and in particular a
rational formula for
the PageRank in terms of
[3].
The implications of the
result are twofold. From a theoretical viewpoint, a general
theorem on standard canonical forms of special rank-one
perturbations is suggested [2]. From a
practical
viewpoint, the use of extrapolation procedures [1]
could be the key point for the efficient computation of the
PageRank when
is close or equal to
.
BiCGStab, the stabilised biconjugate gradient algorithm [8], is Van der Vorst's popular and successful method for the solution of a large sparse system of linear equations. Usually, the system is expressed as
Brezinski and Redivo-Zaglia [3] have found extensions of BiCGStab which avoid serious breakdowns; other possible modifications of BiCGStab have been reviewed by Gutknecht, [6].
There is another, closely related algorithm algorithm called VPAStab
(stabilised vector-Padé approximation) which can
also be applied to the solution of large sparse systems of linear
equations, [4,9].
VPAStab was devised
to provide a staircase sequence of vector-valued rational approximants
for partial sums of the series in powers of
of
To end with a familiar theme, [1,2], we will recall that BiCGStab developed from Lanczos' BiCG 3-term recursion relations, whereas VPAStab developed from Frobenius' 3-term recursion relations. The similarities between these identities are well-known here in Lille, and their derivations can be found from the determinantal formulas for the polynomials in question.
A variety of iterations exist
for computing matrix functions
such as the sign function, matrix roots, and the unitary polar factor.
Underlying many of these iterations are Padé approximants
to an appropriate scalar function.
We review the role of Padé approximation in iterations for
computing the
matrix sign function and the matrix square root.
We show how questions of convergence and stability
of iterations for these functions
can often be reduced to
consideration of whether the powers of a certain matrix converge to
zero,
or are bounded.
The analysis of stability is formalized in a new way in terms of
Frechét
derivatives.
Our approach permits short, general, unifying analyses
of convergence and stability that
avoid consideration of the Jordan canonical form.
We also analyze some linearly convergent iterations for the matrix
square
root.
A different approach to analyzing convergence is now necessary,
and in one case an unexpected connection is found with the Mandelbrot
set.
[Return to schedule]
During the last ten years there has been a renewed interest in
computing bounds or
estimates of norms of the errors in the Conjugate Gradient algorithm
for solving linear systems with
symmetric positive definite matrices, see [3], [6], [8] and in iterative
methods for more general systems
[2].
The techniques used in [4], [6]
are based on writing the norms of the error (A or l2
norms)
as Riemann-Stieltjes integrals and to obtain bounds or estimates of
these integrals with
Gauss quadrature rules. The Gauss rule gives a lower bound. If we have
estimates of the
smallest and largest eigenvalues of A, we can use the
Gauss-Radau and Gauss-Lobatto
quadrature rules which can provide upper bounds. Such estimates can be
used to devise
reliable stopping criteria for stopping the iterations, particularly in
problems arising from the
finite element method, see [1]. It has been shown in
[5], [9] that these techniques
can be
efficiently used in finite precision computations despite the rounding
errors that delay CG
convergence.
In this lecture we shall summarize these techniques, describe their
relations with orthogonal
polynomials and show how they can be extended to the Preconditioned
Conjugate Gradient, see [7], [10].
We shall provide several numerical examples of the effectiveness of the
estimates of the error
norms.
[1] M. Arioli, Stopping criterion for the conjugate gradient algorithm in a finite element method framework, Numer. Math., v 97, (2004), pp 1-24.
[2] C. Brezinski, Error estimates in the solution of linear systems, SIAM J. Sci. Comput., v 21 n 2, (1999), pp 764-781.
[3] G.H. Golub and G. Meurant, Matrices, moments and quadrature, in Numerical Analysis 1993, D.F. Griffiths & G.A. Watson Eds., Pitman Research Notes in Mathematics, v 303, (1994), pp 105-156.
[4] Golub and G. Meurant, Matrices, moments and quadrature II or how to compute the norm of the error in iterative methods, BIT, v 37 n 3, (1997), pp 687-705.
[5] G.H. Golub and Z. StrakoŠ, Estimates in quadratic formulas, Numerical Algorithms, v 8 no II-IV, (1994).
[6] G. Meurant, The computation of bounds for the norm of the error in the conjugate gradient algorithm, Numerical Algorithms, v 16, (1997), pp 77-87.
[7] G. Meurant, Numerical experiments in computing bounds for the norm of the error in the preconditioned conjugate gradient algorithm, Numerical Algorithms, v 22, (1999), pp 353-365.
[8] G. Meurant, The Lanczos and Conjugate Gradient algorithms, from theory to finite precision computations, to be published by SIAM, (2006).
[9] Z. StrakoŠ and P. TichÝ, On error estimation in the conjugate gradient method and why it works in finite precision computation, Electr. Trans. Numer. Anal., v 13, (2002), pp 56-80.
[10] Z. StrakoŠ and P. TichÝ, Error estimation in preconditioned conjugate gradient, accepted in BIT Numerical Mathematics, (2005).
A new algorithm is proposed for computing the intersection of two plane curves assigned in a rational parametric form. It relies on the Ehrlich-Aberth iteration complemented with some computational tools like the properties of Sylvester and Bézout matrices, a stopping criterion based on the concept of pseudo-zero, an inclusion result and the choice of initial approximations based on the Newton polygon. The algorithm is implemented as a Fortran 95 module. From the numerical experiments performed with a wide set of test problems it turns out its superiority with respect to the Manocha-Demmel approach based on eigenvalue computation. In fact, the algorithm provides better approximations in terms of the relative error and performs successfully in many critical cases where the eigenvalue computation fails.
Research in numerical linear algebra is changing rapidly because of the new trends in the sciences and the world economy. For example, the current information revolution has given rise to a large number of new challenges to computer scientists and mathematicians. Many of the numerical methods in scientific computing of the past few decades were motivated by problems arising from the solution of PDEs -- as a response to demands of now established industries, such as the aerospace industry (Navier-Stokes). A few of the new trends that are likely to shape the decades ahead are: (1) Nanotechnology (e.g. materials science), (2) Biology and genetics (3) information technology. In each case there are enormously challenging problems to solve which require matrix algorithms that are far more powerful than those available today. I will discuss a few of these problems. In materials science, the fundamental equation is Schrodinger and the basic problem is an eigenvalue problem with many eigenvalues. In genetics, one of the major issues is clustering, i.e., recognizing patterns in genes. In information sciences, one can mention again clustering (e.g., face recognition) but also the broad problem of dimensionality reduction (e.g., Principal Component Analysis) which is essential for handling huge data sets.
Many environmental studies rely on modelling geochemical reactions combined with hydrodynamic processes such as groundwater flow, transport of solutes by advection and diffusion, heat transfer in porous media. Some issues concern aquifer contamination, underground waste disposal, etc. Reactive transport models are complex non-linear PDEs, coupling the transport engine with the geochemical operator. We discuss efficient and robust numerical methods, based on DAE solvers, combined with either a Gauss-Seidel or on a modified Newton method with a powerful linear solver.
The evaluation of
In many of the applications mentioned above the matrix
is
large and sparse or structured, typically resulting from
discretization of an infinite-dimensional operator. In this case
evaluating (4) by first computing
is usually
unfeasible, so that most of the algorithms for the latter task
cannot be used. The standard approach for approximating
(4) directly is based on a Krylov subspace
of
with initial vector
or, more precisely,
on its Arnoldi decomposition
It is well known that the vector
of (5)
can also be represented as
To judge the quality of
as an approximation to
we thus consider the following two problems:
The classical approach to answer the first question is based on
potential theory and, as recent results of Kuijlaars and
Beckermann show, the latter theory also helps to attack the second
problem (at least if
is a normal matrix).
Based on these
observations we present a qualitative analysis of the convergence
properties of Krylov subspace approximations to
. Special
emphasis is put on the question under which assumptions on
and
we can expect a superlinear convergence
behavior.
Multivariate polynomial interpolation formulae which are extension of
the most well-known ones for univariate polynomial interpolation are
revisited. Some natural ways to obtain them show the relation with
elimination techniques, totally positive matrices and other branches of
mathematics.
[Return to schedule]
This talk will discuss computational algorithms that deal with the
following general task: given a function f and a dictionary of
functions D in a Hilbert
space, extract a linear combination of N functions in D which
approximates
f at best. We shall review the properties of existing algorithms, and
establish some
new convergence properties. This work is motivated by applications as
various
as data compression, adaptive numerical simulation of PDE's in large
space dimension, statistical learning theory.
[Return to schedule]
This paper is the continuation of a work initiated in [3] about the
computation of Hermite-Padé forms (HPF) and associated
Hermite-Padé approximants (HPA) to the exponential function. We
present an alternative algorithm also based on the representation of
HPF, which are divided differences of the function
, in terms of integral remainders
with B-splines as Peano kernels. This algorithm uses the nice
properties of discrete B-splines giving rise to a great variety of
representations of HPF of higher orders in terms of HPF of low orders,
and in particular of classical Padé forms. We give some
illustrations of this algorithm, for example we find again quadratic
HPF of the exponential functions already described in [1] and [2].
Extensions to other types of functions are also possible.
[1] P. Borwein : Quadratic HPA to the exponential function. Constr. Approx. 2 (1986) 291-302.
[2] K. A. Driver : Nondiagonal quadratic HPA to the exponential function. J. Comput. Appl. Math. 65 (1995) 125-134.
[3] P. Sablonnière : An algorithm for the computation of HPA
to the
exponential function: divided differences and HPF. Numer. Algorithms 33
(2003)
443-452.
[Return to schedule]
See the pdf file.
The recurrence coefficients of certain semi-classical orthogonal
polynomials satisfy
discrete Painlevé equations. The Freud equation for the
recurrence coefficients
of the orthogonal polynomials for the weight
is in fact
a special case of discrete Painlevé I, the Verblunsky
coefficients of orthogonal
polynomials on the unit circle with weight
satisfy
discrete Painlevé II, the recurrence coefficients of generalized
Charlier polynomials
can be written in terms of a solution of discrete Painlevé II,
and a
-deformation
of the Freud polynomials on the exponential lattice
has recurrence coefficients that satisfy a
-discrete Painlevé I equation.
Unfortunately, these non-linear recurrence relations are not suited for
computing
the recurrence coefficients starting from two initial conditions, since
minor
deviations from the correct initial values quickly leads to major
deviations from the
correct value. For the Freud equations for the weight
, Lew
and Quarles showed that there is a unique solution of the
Painlevé I equation
which starts at 0 and remains positive for all
. This positive solution is in fact
a fixed point in a metric space of sequences, and it can be found by
successive
iterations of a contractive mapping. This procedure give a numerically
stable way to
compute the recurrence coefficients. We will show that a similar result
is also true
for the Painlevé II equation and for the
-discrete Painlevé I equation. In both
cases the fixed point solution is precisely the solution that gives the
recurrence
coefficients of the corresponding orthogonal polynomals.
In a letter of February 16, 1734 to his friend
Daniel Bernoulli, Euler reports on his attempt to compute the
common logarithm by interpolation at the successive powers of
ten. He notes that for log 9 the procedure, though converging
fast, yields an incorrect answer. The interpolation procedure
is analyzed mathematically, and the discrepancy explained on
the basis of modern function theory. Alternative procedures,
also based on interpolation, are suggested that in principle
yield more satisfactory answers.
[Return to schedule]
In our talk we will survey some applications of non-standard
orthogonal polynomials in Numerical Analysis and Approximation
Theory.
I. Spectral transformations for Hermitian Toeplitz matrices
Let
be a nontrivial probability measure
supported on the unit
circle and let denote
![]() |
(6) |
(i)
(ii)
,
(iii)
,
.
The matrix representation of the multiplication operator in terms of
the above orthogonal polynomial basis, the so called GGT
representation, is a Hessenberg matrix. We will present some recent
results (see [2]) concerning the LU and
QR
factorization of such a matrix and the connection with some
canonical spectral transformations of
which
are the
counterpart on the unit circle of the Geronimus, Christoffel, and
Uvarov transforms for measures supported on the real line, i.e, the
spectral measures of Jacobi matrices (see [1]).
On the
other hand, we will analyze similar factorization problems for the
five-diagonal representation of the multiplication operator, the so
called CMV representation, where an orthogonal basis of Laurent
polynomials with respect to
is needed.
II.Sobolev orthogonal polynomials
Let
be a vector of positive Borel
measures
supported on the real line. In the linear space
of polynomials
of real coefficients we introduce an inner product
An application of such Sobolev orthogonal polynomials to the solution of semilinear two-point boundary value problems using a Galerkin method will be described (see [4]).
In the case
and
for positive Borel
measures with unbounded support very few examples were considered in
the literature. Nevertheless, a general approach is given in
[6]. There the authors assume
and
.
If
denotes the sequence of orthonormal
polynomials with
respect to the Sobolev inner product and
is
the sequence
of orthonormal polynomials with respect to
, then
behaves like
for fairly general weight on a unbounded interval but some growth
restrictions on
are needed. If
grows too fast at
, then the first term in the inner product will
swamp the
term involving the derivative. As a sample, the Freud weight
where
is an
continuous function in
willbe
studied. Assuming some extra conditions on
, the weighted
estimate of
in
and
as well as strong asymptotics for
the re-scaled polynomials are obtained.
An interesting problem is the comparison of the zero distribution of
and
. In
particular, an estimate of the largest
zero of
in terms of n will be discussed.
Finally, the behavior for smooth functions of the Fourier expansions
with respect to
and
will be analyzed.
A tolerant derivative-free nonmonotone line search technique is proposed and analyzed. As a globalization strategy, it allows several consecutive increases in the objective function and also non descent directions for unconstrained minimization. To illustrate the power of this new line search we describe a direct search algorithm in which the directions are chosen randomly. The convergence properties of this random method relies exclusively on the line search technique. We also present numerical experiments with the spectral gradient method, and the inverse SR1 update that could produce non descent directions. In both cases we report results using the exact gradient vector and also a local variation finite differences approximation to the gradient.