An object-based representation for video data can facilitate video search and content analysis. Detecting physical meaningful video object is a challenging open issue, and requires intelligent spatio-tempral segmentation and tracking. Normally,this is done through spatio- temporal segmentation and region tracking. In this work, some of the practical issues of segmentation and tracking problems are addressed. Due to the limitation of using low-level visual features in the segmentation, the tracked regions are more likely to be fragmented parts of some meaningful objects. However, if a collection of video shots that contain a particular object of interest are given, spatio-temporal correlations would exist between the neighboring regions of the object. A method of minig association rules is used to discover these patterns and thus to find possible objects in the scene. Initial experimental results of this approach are shown.