TY - JOUR

T1 - A feature-based approach for dense segmentation and estimation of large disparity motion

AU - Wills, Josh

AU - Agarwal, Sameer

AU - Belongie, Serge

N1 - Funding Information: We would like to thank Charless Fowlkes and Ben Ochoa for helpful discussions. The images in Figs. 13 and 14 are used courtesy of Dr. Philip Torr. This work was partially supported under the auspices of the U.S. Department of Energy by the Lawrence Livermore National Laboratory under contract No. W-7405-ENG-48, by an NSF IGERT Grant (Vision and Learning in Humans and Machines, #DGE-0333451), by an NSF CAREER Grant (Algorithms for Nonrigid Structure from Motion, #0448615) and by The Alfred P. Sloan Research Fellowship.

PY - 2006/6

Y1 - 2006/6

N2 - We present a novel framework for motion segmentation that combines the concepts of layer-based methods and feature-based motion estimation. We estimate the initial correspondences by comparing vectors of filter outputs at interest points, from which we compute candidate scene relations via random sampling of minimal subsets of correspondences. We achieve a dense, piecewise smooth assignment of pixels to motion layers using a fast approximate graphcut algorithm based on a Markov random field formulation. We demonstrate our approach on image pairs containing large inter-frame motion and partial occlusion. The approach is efficient and it successfully segments scenes with inter-frame disparities previously beyond the scope of layer-based motion segmentation methods. We also present an extension that accounts for the case of non-planar motion, in which we use our planar motion segmentation results as an initialization for a regularized Thin Plate Spline fit. In addition, we present applications of our method to automatic object removal and to structure from motion.

AB - We present a novel framework for motion segmentation that combines the concepts of layer-based methods and feature-based motion estimation. We estimate the initial correspondences by comparing vectors of filter outputs at interest points, from which we compute candidate scene relations via random sampling of minimal subsets of correspondences. We achieve a dense, piecewise smooth assignment of pixels to motion layers using a fast approximate graphcut algorithm based on a Markov random field formulation. We demonstrate our approach on image pairs containing large inter-frame motion and partial occlusion. The approach is efficient and it successfully segments scenes with inter-frame disparities previously beyond the scope of layer-based motion segmentation methods. We also present an extension that accounts for the case of non-planar motion, in which we use our planar motion segmentation results as an initialization for a regularized Thin Plate Spline fit. In addition, we present applications of our method to automatic object removal and to structure from motion.

KW - Graph cuts

KW - Layer-based motion

KW - Markov Random Field

KW - Metric labeling problem

KW - Motion segmentation

KW - Periodic motion

KW - RANSAC

UR - http://www.scopus.com/inward/record.url?scp=33646575547&partnerID=8YFLogxK

U2 - 10.1007/s11263-006-6660-3

DO - 10.1007/s11263-006-6660-3

M3 - Journal article

AN - SCOPUS:33646575547

VL - 68

SP - 125

EP - 143

JO - International Journal of Computer Vision

JF - International Journal of Computer Vision

SN - 0920-5691

IS - 2

ER -