An Imbalance Compensation Framework for Background Subtraction

Class imbalance refers to the instance where the number of training samples for the majority classes is far more than that of the minority classes (relative imbalance), and the quality of training samples for the minority classes is inferior to that of the majority classes (absolute imbalance), which are further complicated by other imbalance factors, e.g., data overlapping. Video background subtraction aims to classify each pixel into two classes: foreground and background. This paper first reveals that background subtraction is a class imbalance problem, where the foreground and background are the minority and majority classes, respectively. By exploring spatial and temporal correlation inherent in video data, we present an imbalance compensation framework for background subtraction, which consists of two sequential modules, imbalance-compensated bilayer modeling, and imbalance-compensated Bayesian classification. In the first module, spatio-temporal oversampling (SOS) and selective downsampling (SDS) are proposed to compensate the imbalance at data level. SOS attempts to synthesize representative samples appended to the minority sample set, while SDS selectively deletes a number of majority samples in data overlapping areas. The rebalanced samples are then used to learn a bilayer model. In the second module, novel cost functions are proposed to compensate the effect of class imbalance at algorithm level. The cost functions are based on imbalance measurement, and used to construct the prior term in the Bayesian classification scheme. Experiments are conducted on public databases to demonstrate the effectiveness of the proposed method.