A General Theory of Convergence for Numerical Methods

The concepts of convergence, consistency and stability are given very general definitions that are applicable to any numerical computation. The main feature of these definitions is the treatment of truncation error and rounding error as special cases of the unifying concept of a perturbation. Thus stability, which has to do with rounding errors, and consistency, which has to do with truncation errors, are both expressed in terms of perturbations. This gives an abstract version of the Lax theorem (consistency + stability = convergence) a very simple proof, applicable to any numerical method. The conceptual identification of the two sources of error also yields a general theory of the order of convergence of an algorithm in the presence of rounding errors. It is proved that the order of convergence is the same as the order of consistency if the method has a property called order-stability. The usefulness of the theory is demonstrated by analyses of methods for solving both initial value and boundary value problems in ordinary differential equations, where it is found that order-stability and stability are equivalent. The theory does not obviate the need for any of the detailed analysis normally required, but the insight it provides enables us to extend the traditional results to include differential equations $y' = f(x,y(x))$ in which f is a discontinuous function of its first argument.