Konzeptpropagierung mittels einer partiellen Ähnlichkeitssuche für Bilder im World-Wide-Web
Final Report Abstract
Current web search engines link images with the surrounding text, therefore allowing access to the image database only through text-based queries. Such an approach restricts significantly the set of searchable images, and is also susceptible to errors, since no assessment of the relevance of the textual information associated to each image is made. Alternative procedures relying on image processing techniques have limited usefulness, because the developed methodologies either have restricted application fields or rely on nonspecific image features with low interpretability. This project aimed to integrate such different image processing viewpoints into a more efficient framework for automatic image annotation, search and retrieval. In the preliminary stages, we have implemented algorithms for the extraction of interest points from image files and of keywords from the associated text, which were subsequently employed in the development of a training dataset based on news articles collected from the web. Preliminary exploratory investigations showed that it is indeed possible to establish a correspondence between the semantic similarity of articles and the visual similarity of the associated images. Further experiments have focused on the quantification and exploitation of such intrinsic text-image associations. In particular, a system for the annotation of novel images based on their visual information and on the text associations of similar training images has been successfully demonstrated in a restricted setting. A more thorough evaluation of the developed methodology in a generalized setting has been however delayed by computational power limitations. Therefore, we do not envisage an immediate commercial application, but we consider that the final prototype would be an efficient complementary research tool in mass-media-related fields.
Publications
- (2007). Automatic image annotation by association rules. Electronic Imaging and the Visual Arts EVA 2007, (pp. 108-112). Berlin, Germany
Jacobs, A., Hermes, T., & Wilhelm, A.
- Semantic Video Segmentation using Probabilistic Relaxation. In International Workshop on Visual and Multimedia Digital Libraries, Modena, Sept. 13-14, 2007
A. Jacobs & G. T. Ioannidis
- (2008). Inter-video similarity for video parsing. In Proc. 5th IFIP International Conference on Intelligent Information Processing, October 19-22, 2008, Beijing, China (pp. 174-181). IFIP - International Federation for Information Processing. Springer: Boston
A. Jacobs, A. Lüdtke & O. Herzog
- (2008). Projection-based clustering for high-dimensional data sets. In COMPSTAT 2008: Proceedings in Computational Statistics. Heidelberg, Germany: Physica Verlag
Ilies, I., & Wilhelm, A.
- (2008). Relaxation-based data mining on images and text from news web sites. Proceedings of IASC2008, (pp. 736-743). Yokohama, Japan
Jacobs, A., Herzog, O., Wilhelm, A., & Ilies, I.
- (2010). Association rule mining of multimedia content. In F. Palumbo, C. N. Lauro, & M. J. Greenacre (Eds.), Data Analysis and Classification. Berlin, Germany: Springer
Wilhelm, A., Jacobs, A., & Hermes, T.
- (2010). Cluster analysis for large, high-dimensional datasets: Methodology and applications. Doctoral dissertation, Jacobs University Bremen, Germany
Ilies, I.
- (2010). Combining text and image processing in an automated image classification system. In Computing Science and Statistics: Proceedings of the 2010 Symposium on the Interface. Seattle, WA
Ilies, I., Jacobs, A., Herzog, O., & Wilhelm, A.
- (2010). Projection-based partitioning for large, high-dimensional datasets. Journal of Computational and Graphical Statistics, 19(2), 474-492
Ilies, I., & Wilhelm, A.
- (2010).Ein deformationsinvarianter Point-of-Interest-Detektor. Doctoral dissertation, Bremen University, Germany
Jacobs, A.
