We analyze and evaluate different initialization methods for motion layers segmentation, a powerful mid-level vision tool. We estimate 6-D affine motion models from the optical flow corresponding to different over-segmentations. Over-segmentations can be either uniform rectangular blocks; or adaptive sized rectangular blocks; or super-pixels; or based on any other clustering methods. We present performance analysis of motion segmentation initialization algorithms on video sequences and discuss the relative pros and cons of these methods in terms of hardware implementation issues.