VRL an eye

SpiritTagger: Large-Scale Image Annotation for Worldwide Photographs By Mining Flickr

Objective

SpiritTagger is an image annotation system built to explore knowledge extraction via mining of millions of photographs taken worldwide referenced with a geographical coordinate. We address research problems in fast retrieval of image data, location estimation, descriptor usage, annotation methods and automatic learning tag semantics.

SpiritTagger v.0.2

SpiritTagger Flow

SpiritTagger annotates or "tags" new images anywhere in the world throught inference from a database of 1.4 million images. A list of ranked tag suggestions is generated by considering visual similarity constrained by a geographic region. By representing the data efficiently using a quadtree, the suggestions are generated within a matter of seconds.

In this research we attempt to show how to employ visual descriptors in an intelligent fashion over a spatial database so as to generate more accurate automatic image annotations. We employ an unsupervised data-centric approach, rather than one relying on trained models, in order to take advantage of emergent web repositories.

People

Jim Kleban, Emily Moxley, Jiejun Xu and B.S.Manjunath

Demo

» SpiritTagger Demo

Downloads

Coming shortly...

Acknowledgements

Thanks to Dmitry Fedorov and Kris Kvilekval for their help and wide range of knowledge. Work supported by NSF IGERT.

Publications

    2009

  1. J. Kleban, E. Moxley, J. Xu, and B. S. Manjunath,
    "Global Annotation of Georeferenced Photographs"
    Proceedings of ACM Conference on Image and Video Retrieval, Greece, Jul. 2009.
    [abstract] [BibTex]

    Abstract preview: "We present an efficient world-scale system for providing automatic annotation on collections of geo-referenced photos. As a user uploads a photograph a place of origin is estimated from visual feature..." [more]

  2. E. Moxley, J. Kleban, J. Xu, and B. S. Manjunath,
    "Not All Tags Are Created Equal: Learning Flickr Tag Semantics for Global Annotation."
    Proceedings of IEEE International Conference on Multimedia and Expo, New York, Jun. 2009.
    [abstract] [BibTex]

    Abstract preview: "Large collaborative datasets offer the challenging opportunity of creating systems capable of extracting knowledge in the presence of noisy data. In this work we explore the ability to automatically l..." [more]

  3. J. Kleban, E. Moxley, and B. S. Manjunath,
    "SpiritTagger: A Location-Aware Interactive Web Tool for Annotating Your Photo Collection."
    The Future of Interactive Media: Workshop on Media Arts, Science, and Technology (MAST), Santa Barbara, CA, Jan. 2009.
    [abstract] [BibTex]

    Abstract preview: "Digital photos are an important aspect of the modern multimedia experience. The contributions of online collaborative communities can now be leveraged for the task of automated image annotation. We pr..." [more]

    4. 2008

  4. Emily Moxley, Jim Kleban and B.S. Manjunath,
    "SpiritTagger: A Geo-Aware Tag Suggestion Tool Mined from Flickr"
    Proc. ACM International Conference on Multimedia Information Retrieval (MIR2008), Vancouver, Canada, Oct. 2008.
    [abstract] [PDF] [BibTex]

    Abstract preview: "Geographically referenced, or "geo-tagged", photo data sets offer tantalizing potential for automated knowledge discovery in the world. By combining tag reranking based on geographic context with cont..." [more]