Image stabilization and mosaicking using the overlapped basis optical flow field

Image stabilization and image mosaicking are fundamental image sequence operations that compensate for camera motion. Stabilization is a precursor to background deletion, region of interest extraction, motion discontinuity estimation and video compression. The effectiveness of an end-to-end algorithm is closely tied to the stabilization accuracy of the input sequence, which in turn depends upon (i) the accuracy of the estimated motion in the scene, (ii) the choice and compliance of the global motion model and (iii) the camera calibration. We deal with (i) and demonstrate the merits of a robust technique for computing optical flow, using overlapped basis functions proposed by Srinivasan and Chellappa (see CAR-TR-845, Univ. of Maryland, 1996). In this technique, we regularize the ill-conditioned gradient constraint equation by modeling optical flow as a linear combination of an overlapped set of basis functions. The solution, which can be shown to be optimal, is obtained by a numerically stable sparse matrix inversion, giving a reliable flow field estimate over a large fraction of the frame. We employ an iterated least squares method for consolidating the local model parameters into a global model, for which we choose a six parameter affine transformation. We argue that our flow field model offers a higher accuracy and robustness than conventional optical flow techniques, and is well suited for image stabilization and mosaicking.