Human segmentation by fusing visible-light and thermal imaginary

This paper describes a system for robust segmentation of human in video sequences by fusing the visible-light and thermal imaginary. The system first performs a simple calibration procedure to rectify the two camera views without knowing the cameras' intrinsic characteristics. Then a blob-to-blob homography is learned on-the-fly by estimating the disparity of each blob so that a pixel level registration can be achieved. The multi-modality information is then combined under a two-tier tracking algorithm and a unified background model to attain precise segmentation. Preliminary experimental results shows significant improvements over existing schemes under various difficult scenarios.