Rolling shutter distortion correction

As opposed to the global shutter, which starts and stops the light integration of each pixel at the same time by incorporating a sample-and-hold switch with analog storage in each pixel, the electronic rolling shutter found in most low-end CMOS image sensors today collects the image data row by row, analogous to an open slit that scans over the image sequentially. Each row integrates the light when the slit passes over it. Therefore, the scanlines of the image are not exposed at the same time. This sensor architecture creates an objectionable geometric distortion, known as the rolling shutter effect, for moving objects. In this paper, we address this problem by using digital image processing techniques. A mathematical model of the rolling shutter is developed. The relative image motion between the moving objects and the camera is determined by block-based motion estimation. A Bezier curve fitting is applied to smooth the resulting motion data , which are then used for the alignment of scanlines. The basic ideas behind the algorithm presented here can be generalized to deal with other complicated cases.