Video Segmentation with Motion Smoothness

Abstract
In this paper, we present an interactive and intuitive graph-cut-based video segmentation system while taking both color and motion information into consideration with a stroke-based user interface. Recently, graph-cut-based methods become prevalent for image and video segmentation. However, most of them deal with color information only and usually failed under circumstances where there are some regions in both foreground and background with similar colors. Unfortunately, it is usually hard to avoid, especially when the objects are filmed under a natural environment. To make such methods more practical to use, we propose a graph-cut-based video segmentation method based on both color and motion information, since the foreground objects and the background usually have different motion patterns. Moreover, to make the refinement mechanism easy to use, the strokes drawn by the user are propagated to the temporal-spatial video volume according to the motion information for visualization, so that the user can draw some additional strokes to refine the segmentation result in the video volume. The experiment results show that by combining both color and motion information, our system can resolve the wrong labeling due to the color similarity, even the foreground moving object is behind an occlusion object.

Citation
Chung-Lin Wen, Bing-Yu Chen, and Yoichi Sato. Video Segmentation with Motion Smoothness. IEICE Tran. on Information and Systems E93-D(4), 2010
Chung-Lin Wen, Yu-Ting Wong, Bing-Yu Chen, and Yoichi Sato. Video Segmentation with Motion Smoothness. ACM SIGGRAPH 2009 Conference Abstracts and Applications (Posters), Article 40, New Orleans, Louisiana, USA., 2009.

Paper
IEICE 2010 Paper (904KB PDF)
ACM SIGGRAPH 2009 Poster (193KB PDF)

Video

Last update: by Chung-Lin Wen