Publications

We describe recent research into using the visual primitive of texture to analyze and manage large collections of remote sensed image and video data. Texture is regarded as the spatial dependence of pixel intensity. It is characterized by the amount of dependence at different scales and orientations, as measured with frequency-selective filters. A homogeneous texture descriptor based on the filter outputs is shown to enable (1) content-based image retrieval in large collections of satellite imagery, (2) semantic labeling and layout retrieval in an aerial video management system, and (3) statistical object modeling in geographic digital libraries.

Object based representations of image data enable new content-related functionalities while facilitating management of large image databases. Developing such representations for multi-date and multi-spectral images is one of the objectives of the second phase of the Alexandria Digital Library (ADL) project at UCSB. Image segmentation and image registration are two of the main issues that are to be addressed in creating localized image representations. We present in this paper some of the recent and current work by the ADL’s image processing group on robust image segmentation, registration, and the use of image texture for content representation. Built upon these technologies are techniques for managing large repositories of data. A texture thesaurus assists in creating a semantic classification of image regions. An object-based representation is proposed to facilitate data storage, retrieval, analysis, and navigation. image content descriptors. Perhaps the most noticeable result is the upcoming MPEG-7 standardization effort whose objective is to provide a set of standardized tools to describe multimedia content. However, the expectations placed on the ability of these descriptors to capture meaningful information must be tempered. The human visual system requires extensive learned experience to function as well as it does. This is especially true for an expert viewing a restricted domain of images.

Our contribution in this paper is two-fold. First, we extend our previous curve evolution method based on pair-wise similarities. This curve evolution equation combines the grouping abilities of active contours and graph partitioning techniques. Connections of our method to spectral graph partitioning are investigated and comparisons are made. Second, in a model-based segmentation scenario, we propose a method to improve segmentation quality by iteratively modifying the model using feed-back from segmentation of a labeled training set. Our purpose here is to segment objects in geo-spatial images by integrating domain knowledge with the segmentation method. We achieve our goal by combining a statistical model for the object with a knowledge-guided segmentation method. Experimental results show that this framework is effective for model- based segmentation of complex geo-spatial objects.

In this paper, we propose an affine-invariant method for describing and matching curves. This is important since affne transformations are often used to model perspective distortions. More specically, we propose a new definition of the shape of a curve that characterizes a curve independently of the effects introduced by a±ne distortions. By combining this definition with a rotation-invariant shape descriptor, we show how it is possible to describe a curve in an intrinsically affine- invariant manner. To validate our procedure we built a database of shapes subject to perspective distortions and plotted the precision-recall curve for this dataset. Finally an application of our method is shown in the context of wide baseline matching.

Texture remains largely underutilized in the analysis of remote sensed datasets compared to descriptors based on the orthogonal spectral dimension. This paper describes our recent efforts towards using texture to automate the annotation of remote sensed imagery. Two applications are described that use the homogeneous texture descriptor recently standardized by MPEG-7. In the first, higher-level access to remote sensed imagery is enabled by using the texture descriptor to model geo-spatial objects. In particular, the common textures, or texture motifs, are characterized as Gaussian mixtures in the high-dimensional feature space. In the second application, the texture descriptor is used to label regions in a large collection of aerial videography in a perceptually meaningful way. Gaussian mixtures are used to model the distribution of feature vectors for a variety of semantic classes. Frame level similarity retrieval based on semantic layout and semantic histogram is enabled by modeling the spatial arrangement of the labeled regions as a Markov random field.

Effectiveness and efficiency are two important concerns in using multimedia descriptors to search and access database items. Both are affected by the dimensionality of the descriptors. While higher dimensionality generally increases effectiveness, it drastically reduces efficiency of storage and searching. With regard to effectiveness, relevance feedback is known to be a useful tool to squeeze information from a descriptor. However, not much has been done toward enabling relevance feedback computation using high-dimensional descriptors over a large multimedia dataset. In this context, we have developed new methods that enable us to a) reduce the dimensionality of Gabor texture descriptors without losing on effectiveness, and b) perform fast nearest neighbor search based on the information available during each iteration of a relevance feedback step. Experimental results are presented on real datasets.

Texture has been recognized as an important visual primitive in image analysis. A widely used texture descriptor, which is part of the MPEG-7 standard, is that computed using multiscale Gabor filters. The high dimensionality and computational complexity of this descriptor adversely affect the efficiency of content-based retrieval systems. We propose a modified texture descriptor that has comparable performance, but with nearly half the dimensionality and less computational expense. This gain is based on a claim that the distribution of (absolute values of) filter outputs have a strong tendency to be Rayleigh. Experimental results show that the dimensionality can be reduced by almost 50%, with a tradeoff of less than 3% on the error rate. Furthermore, it is easy to compute the new feature using the old one, without having to repeat the computationally expensive filtering step. We also propose a new normalization method that improves similarity retrieval and indexing efficiency.

This work presents a novel approach to object localization in complex imagery. In particular, the spatial extents of objects characterized by distinct spatial signatures at multiple scales are estimated by using statistical models to control a simple region growing process. Texture motifs are used to model the spatial signatures at the smallest, or pixel, scale. Markov random fields are used to model the spatial signatures at the larger, or motif, scale. These models are used to iteratively expand a bounding box to approximate the spatial extent of an object. The approach is applied to localizing geo-spatial objects in high-resolution panchromatic aerial imagery.

We propose a canonical model for object classes in aerial images. This model is motivated by the observation that geographic regions of interest are characterized by collections of texture motifs corresponding to the geographic processes that generate them. We show that this model is effective in learning the common texture themes, or motifs, of the object classes.

We propose a canonical model for object classes in aerial images. This model is motivated by the observation that geographic regions of interest are characterized by collections of texture motifs corresponding to the geographic processes that generate them. We show that this model is effective in learning the common texture themes, or motifs, of the object classes.

What are the natural features of hand-written characters and how to arrive at them automatically? We apply independent components analysis on hand-written characters. Independent components analysis extracts the underlying statistically independent signals from a mixture of them. We expect strokes to be the independent components of hand-written characters. Our findings show that stroke-like features emerge as a result of the analysis confirming the above intuition. This finding is significant since it gives an automatic procedures for extracting stroke-like features from multilingual character data sets. We use these features for handwritten digit recognition using a very simple classifier. The classifier is chosen to be simple so that the quality of input feature set can be evaluated. The recognition results indicate that the features arrived at by independent component analysis are useful.

A prototype basketball video retrieval system is presented in this report. Retrieval is based on the similarity of ball motion in the clip with that in the query. The system uses a query-by-sketch paradigm, where the user provides a sketch of the desired ball trajectory. The video data is pre-processed to make the ball motion invariant to camera translation. The next stage is dimensionality reduction wherein we model the ball motion as a set of parabolic trajectories. An R-tree is used to index these parabolic representations and search for similar trajectories in a low dimension parametric space. The query is processed to obtain its parametric representation, and a nearest neighbor search is performed for similar parabolas. These query results are then post- processed by assigning scores based on various similarity criteria. The system could be extended to other types of videos and moving objects. As a proof of concept, the system was tested for ball trajectories in basketball video.