Benchmark for evaluating biological image analysis tools

Elisa Drelie Gelasca, Jiyun Byun, Boguslaw Obara, B.S. Manjunath
Center for Bio-Image Informatics, Electrical and Computer Engineering Department,
University of California, Santa Barbara 93106-9560,


Biological images are critical components for a detailed understanding of the structure and functioning of cells and proteins. Image processing and analysis tools increasingly play a significant role in better harvesting this vast amount of data, most of which is currently analyzed manually and qualitatively. A number of image analysis tools have been proposed to automatically extract the image information. As the studies relying on image analysis tools have become widespread, the validation of these methods, in particular, segmentation methods, has become more critical. There have been very few efforts at creating benchmark datasets in the context of cell and tissue imaging, while, there have been successful benchmarks in other fields, such as the Berkeley segmentation dataset 1, the handwritten digit recognition dataset MNIST 2 and face recognition dataset 3, 4. In the field of biomedical image processing, most of standardized benchmark data sets concentrates on macrobiological images such as mammograms and magnet resonance imaging (MRI) images 5, however, there is still a lack of a standardized dataset for microbiological structures (e.g. cells and tissues) and it is well known in biomedical imaging 5. We propose a benchmark for biological images to: 1) provide image collections with well defined ground truth; 2) provide image analysis tools and evaluation methods to compare and validate analysis tools. We include a representative dataset of microbiological structures whose scales range from a subcellular level (nm) to a tissue level (μm), inheriting intrinsic challenges in the domain of biomedical image analysis (Fig. 1). The dataset is acquired through two of the main microscopic imaging techniques: transmitted light microscopy and confocal laser scanning microscopy. The analysis tools1in the benchmark are designed to obtain different quantitative measures from the dataset including microtubule tracing, cell segmentation, and retinal layer segmentation. Additionally, in the proposed benchmark, ground truth is manually created from part of each dataset. Evaluation methods are provided to evaluate the performance of the analysis tools using the ground truth. The benchmark includes standard evaluation measures and ad hoc methods designed for specific applications. In the following, we briefly explain the evaluation measures used at various scale level.
[PDF] [BibTex]
Elisa Drelie Gelasca, Jiyun Byun, Boguslaw Obara and B.S. Manjunath,
Workshop on Bio-Image Informatics: Biological Imaging, Computer Vision and Data Mining, Santa Barbara, CA, USA, Jan. 2008.
Node ID: 497 , DB ID: 304 , Lab: VRL , Target: Workshop
Subject: [Detection on Images and Videos] « Look up more