Spatial–temporal multi-task learning for salient region detection

This paper proposes a novel multi-task learning based salient region detection method by fusing spatial and temporal features. Salient region detection has been widely used in various computer vision tasks, being as a general preprocessor to identify interest objects. Despite the recent successes, existing saliency models still lag behind the performance of human when visually perceives dynamic scenes. Most of the existing models largely rely on various spatial features. However, these spatial feature based methods have several deficiencies: (i) they can hardly adapt to the situation where moving objects are included, and (ii) they cannot model the human vision process in dynamic scenes. Recently, some saliency models introduce temporal features in their detecting process, such as the optical flow and stacking frames. The potential of temporal features for saliency optimization has been demonstrated. However, since temporal features in these models are merely used as a compensation to static features, the advantages of temporal features have not yet been fully explored. Aiming to comprehensively address these issues above, our method fuses spatial and temporal features, and learns the mapping relationship from various features to salient regions using our multi-task learning framework. The final salient region is generated by our unified Bayesian framework. The experimental results demonstrated that our proposed approach outperforms previous methods.