Abstract: The new video coding standard MPEG4 is enabling content-based functionality. It decomposes sequences into video object planes (VOPs) so that each VOP represents one moving object. This paper presents a new automatic video sequence segmentation algorithm that extracts moving objects. The core of this algorithm is an object tracker that matches a two-dimensional binary model of the object against subsequent frames using the Hausdorff distance. The initial model is derived automatically and a new model update method based on the concept of the moving connected components is proposed. The stationary background is removed by a filtering technique. Finally, the VOPs are extracted from the binary model sequence.