Fusing Keyword Search and Visual Exploration for Untagged Videos
TL;DR
Abstract
Video collections often cannot be searched by keywords because most videos are poorly annotated. We present a system that allows to search untagged videos by sketches, example images and keywords. Having analyzed the most frequent search terms and the corresponding images from the Pixabay stock photo agency we derived visual features that allow to search for 20000 keywords. For each keyword we use several image features to be able to cope with large visual and conceptual variations. As the intention of a user searching for an image is unknown, we retrieve thousands of result images (video scenes), which are shown as a visually sorted hierarchical image map. The user can easily find images of interest by dragging and zooming. The visual arrangement of the images is performed with an improved version of a self-sorting map, which allows organizing thousands of images in fractions of a second. If an image similar to the search query has been found, further zooming will show more related images, retrieved from a precomputed image graph. The new approach helps to find untagged images very quickly in an exploratory, incremental way.
BibTeX
If you use our work in your research, please cite our publication:
@InProceedings{10.1007/978-3-319-73600-6_43,
author="Barthel, Kai Uwe
and Hezel, Nico
and Jung, Klaus",
editor="Schoeffmann, Klaus
and Chalidabhongse, Thanarat H.
and Ngo, Chong Wah
and Aramvith, Supavadee
and O'Connor, Noel E.
and Ho, Yo-Sung
and Gabbouj, Moncef
and Elgammal, Ahmed",
title="Fusing Keyword Search and Visual Exploration for Untagged Videos",
booktitle="MultiMedia Modeling",
year="2018",
publisher="Springer International Publishing",
address="Cham",
pages="413--418",
isbn="978-3-319-73600-6"
}