Abstract

This paper attempts to provide a comprehensive review and characterize the problem of the semantic gap that is the key problem of content-based image retrieval and the current attempts in high-level semantic-based image retrieval being made to bridge it. Major recent publications are included in this review covering different aspects of the research in the area of high-level semantic features. In this paper the different methods of image retrieval systems are described and major categories of the state-of-the-art techniques in narrowing down the ‘semantic gap’ are presented. Finally, based on existing technologies and the demand from real-world applications, a few promising future research directions are suggested.

References

• Riad, A. M., Atwan,A., and Abd El-Ghany,S. 2009. Image Based Information Retrieval Using Mobile Agent, Egyptian Informatics Journal, Vol.10, No.1.
• Kherfi, M.L., Ziou, D., and Bernardi,A. 2004. Image retrieval from the World Wide Web: issues, techniques, and systems. ACM Computing Surveys Vol. 36, No. 1, pp.35–67.
• Riad, A. M., Atwan,A., and Abd El-Ghany,S. 2008. Analysis of Performance of Mobile Agents in Distributed Content Based Image Retrieval. In Proc. IEEE international Conference on Computer Engineering & Systems, ICCES 2008.
• Wang, C., Zhang, L., and Zhang, H. 2008. Learning to Reduce the Semantic Gap in Web Image Retrieval and Annotation. In SIGIR’08, Singapore.
• Alemu, Y., Koh, J., and Ikram, M. 2009. Image Retrieval in Multimedia Databases: A Survey. In Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.
• Men, H. and Chen, J. 2008. A Method of the Extraction of Texture Feature In Proceeding ISICA '08 Proceedings of the 3rd International Symposium on Advances in Computation and Intelligence.
• Liu, Y., Zhang, D., Lu, G., and Ying, W. 2007. A survey of content-based image retrieval with high-level semantics. In Journal of Pattern Recognition Vol. 40, pp: 262– 282.
• Su,J., Wang,B., Yeh, H., and Tseng, V. S. 2009. Ontology-Based Semantic Web Image Retrieval by Utilizing Textual and Visual Annotations. In Web Intelligence/IAT Workshops, pp: 425-428.
• Popescu, A., Grefenstette, G., and Moëllic, P. 2008. Improving Image Retrieval Using Semantic Resources. In Advances in Semantic Media Adaptation and Personalization, pp: 75-96.
• Hou , J., Zhang ,D., Chen ,Z., Jiang, L., Zhang ,H., and Qin ,X. 2010 .Web Image Search by Automatic Image Annotation and Translation. In 17th International Conference on Systems, Signals and Image Processing.
• Saenko,k., Darrell,T. 2008. Unsupervised Learning of Visual Sense Models for Polysemous Word. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems Vancouver, Canada, pp.1393-1400.
• Ménard, E. 2011. Search Behaviours of Image Users: A Pilot Study on Museum Objects. The Canadian Journal of Library and Information Practice and Research, vol. 6, no. 1.
• Smeulders, A.W.M., Member, S., Worring, M., Santini ,S., Gupta ,A., and Jain, R . 2000. Content-based image retrieval at the end of the early years. In IEEE Transactions on Pattern Analysis and Machine Intelligence, pp: 1349–1380.
• Ting, W. Su, Chen, J., Jier, J. and Lien, J. 2010. Region-based image retrieval system with heuristic pre-clustering relevance feedback. In Expert Systems with Applications Vol 37(7): pp:4984-499.
• Shao, L. 2009. An Efficient Local Invariant Region Detector for Image Retrieval. In Canadian Conference on Computer and Robot Vision ,pp 208:212.
• He, R., Xiong ,N., Yang ,L.T., and Park , J. H. 2010. Using Multi-Modal Semantic Association Rules to fuse keywords and visual features automatically for Web image retrieval. In Information Fusion. Vol 12, Issue 3.
• Wong, R.C.F. and Leung, C.H.C. 2008. Automatic Semantic Annotation of Real-World Web Images. IEEE Transactions on Pattern Analysis and Machine Intelligence, VOL. 30, NO. 11, PP 1933-1945.
• He, R., Xiong, N., Kim, T.-H. and Zhu ,Y. 2008. Mining Cross-Modal Association Rules for Web Image Retrieval. 2008. International Symposium on Computer Science and its Applications, pp: 393-396.
• Xu,H., Zhou ,X., Lin ,L., Xiang ,Y., and Shi ,B. 2009.Automatic Web Image Annotation via Web-Scale Image Semantic Space Learning. In Proceedings of the Joint International Conferences on Advances in Data and Web Management, pp. 211–222.
• Zheng, Q.-F., and Gao, W. 2008. Constructing visual phrases for effective and efficient object-based image retrieval. ACM Trans. Multimedia Computing and Communication.
• Chandrika, P., Jawahar, C.V., .2010. Multi Modal Semantic Indexing for Image Retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval, China.
• Huang, R.-B., Dong, S.-L., and Du, M.-H. 2008. A Semantic Retrieval Approach by Color and Spatial Location of Image Regions. In CISP '08, Image and Signal Processing conference, pp: 466- 470, China.
• Nguyen, N.V., Boucher, A., Ogier, J.-M. And TABBONE, S. 2009. Region-based semi-automatic annotation using the Bag of Words representation of the keywords. In Fifth International Conference on Image and Graphics.
• Lin, W.-C., Oakes, M., and Tait, J. 2010.” Improving image annotation via representative feature vector selection”, Neuro computing Vol. 73, pp: 1774–1782.
• Premchaiswadi , W. and Tungkatsathan, A. 2010. On-line Content-Based Image Retrieval System using Joint Querying and Relevance Feedback Scheme. In Journal of World Scientific and Engineering Academy and Society (WSEAS), Vol 9, Issue 5.
• Shen ,H. T. 2009. Speed up interactive image retrieval. In VLDB Journal, Springer, Vol. 18, No. 1, pp: 465-474
• Santos, J. A., Ferreira, C. D., Torres, R., Gonçalves , M. A., and Lamparelli, R. A. C . 2011 .A relevance feedback method based on genetic programming for classification of remote sensing images. In Information Sciences: an International Journal, Vol.181 No.13, pp.2671-2684.
• Zhu, X.S., and Huang, T.S. 2003. Relevance feedback in image retrieval: a comprehensive review”, Multimedia System, pp :536–544.
• Ferreira, C.D., Santos, J.A., Torres, R.S., Gonçalves, M.A., Rezende, R.C., and Fan W. 2011. Relevance Feedback based on Genetic Programming for Image Retrieval. In Pattern Recognition Letters, Vol. 32, Issue 1, pp: 27-37.
• Modaghegh,H., Javidi,M., Yazdi,H.S, and Pourreza, H.R. 2010. Learning of Relevance Feedback Using a Novel Kernel Based Neural Network, Australian Journal of Basic and Applied Sciences, Vol. 4, pp: 171-186.
• Lu, Y., Hu, C., Zhu, X., and Zhang, H., Yang, Q. 2000. A unified framework for semantics and feature based relevance feedback in image retrieval systems. In ACM International Conference on Multimedia, pp. 31–37.
• Chatzilari, .E, Nikolopoulos, S., Papadopoulos. Zigkolis,C., and Kompatsiaris,Y. 2011. Semi-Supervised object recognition using flickr images. In 9th International Workshop on Content-Based Multimedia Indexing, Madrid, Spain.
• Muda, Z. 2008 .Ontological Description of Image Content Using Regions Relationships. In European Semantic Web Conference (ESWC2008), Tenerife, Spain, pp: 46-50.
• Li, Y., Lu, J., Zhang, Y., Li, R., and Xu, W. 2008. Ensemble of Two-class Classifiers for Image Annotation. In International Workshop on Education Technology and Training & International Workshop on Geoscience and Remote Sensing, pp. 763-767.
• Zhang, D., Islam, M.M., Lu, G. 2011. A review on automatic image annotation techniques. In Journal of Pattern Recognition, pp: 1-17.
• Feng, S., and Xu, D. 2010.Transductive Multi-Instance Multi-Label learning algorithm with application to automatic image annotation. In Expert Systems with Applications 37, pp: 661–670.
• Hyvo, E. Styrman , A., and Saarela ,S. 2002. Ontology-based image retrieval. In Proceedings of XML Finland Conference, pp: 27–51.
• Li,Z., Shi,Z. 2009. Modeling latent aspects for automatic image annotation. In 16th IEEE International Conference on Image Processing, Cairo, EGYPT.
• Murdoch, O., Coyle, L. and Dobson, S. 2008. Ontology-based query recommendation as a support to image retrieval. In Proceedings of the 19th Irish Conference in Artificial Intelligence and Cognitive Science.
• Qi,X., and Han,T. 2007. Incorporating Multiple SVMs for Automatic Image Annotation. Pattern Recognition, Vol. 40, No. 2, pp: 728-741.
• Wong, R.C.F. and Leung ,C.H.C. 2008. Automatic Semantic Annotation of Real-World Web Images IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 30, NO. 11, pp 1933- 1944.
• Liua, J., Li, M., Liua Q., Lu, H., and Ma, S. 2009. Image annotation via graph learning. In journal of Pattern Recognition Vol. 42, pp: 218 - 228.
• Liu, X., Ji, R., Yao, H., Xu, P., Sun, X., and Liu, T. 2008 .Cross-Media Manifold Learning for Image Retrieval & Annotation. In Proceedings of the 1st ACM international conference on Multimedia information retrieval (MIR’08), Vancouver, British Columbia, Canada. , pp: 141- 148.
• Udristoiu, S., and Ion, A. L.. 2010 .Image Annotation by Learning Rules from Regions Patterns. In International Conference on Complex, Intelligent and Software Intensive Systems, pp : 124-131.
• Song, H., Li, X., and Wang, P. 2009. Multimodal Image Retrieval Based on Annotation Keywords and Visual Content. In International Conference on Control, Automation and Systems Engineering, pp: 295-289.
• Caudill, J. D. 2009.” BRIDGING THE SEMANTIC GAP IN CONTENT-BASED IMAGE RETRIEVAL. Doctoral Thesis, Department of Computer Engineering and Computer Science University of Louisville, Louisville, Kentucky.
• Kingshy, E., Sychay, G., and GangWu, G. 2003. CBSA: Content-based Soft Annotation for Multimodal Image Retrieval using Bayes Point Machines IEEE Transactions on Circuits and Systems for Video Technology, Vol. 3 ,issue 1, pp: 26 – 38.
• Fan, L., Li, 2007. A User-Driven Ontology Guided Image Retrieval Model. Proceeding COGINF '07 Proceedings of the 6th IEEE International Conference on Cognitive Informatics.
• Fan, L., and Li, B. 2006. A Hybrid Model of Image Retrieval Based on Ontology Technology and Probabilistic Ranking, In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, pp: .477-480.
• Kesorn, K., Poslad, S. 2009. Semantic Restructuring of Natural Language Image Captions to Enhance Image Retrieval. In Journal of Multimedia, Vol. 4, No. 5, pp: 284-297.
• Wang, H., Liu, S., and Chia, L-T. 2006. Does ontology help in image retrieval?: a comparison between keyword, text ontology and multi-modality ontology approaches. In Proceedings of the 14th annual ACM international conference on Multimedia, Santa Barbara, CA, USA.
• Kong, H., Hwang, M., Na, K. and Kim, Pankoo.2005. The Study on the Semantic Image Retrieval Using the Cognitive Spatial Relationships in the Semantic Web. IFIP International Federation for Information Processing.
• Wang, H., Jiang X., Chia and Tan, L.-T. 2008 .Ontology enhanced web image retrieval: aided by wikipedia & spreading activation theory. In Proceeding of the 1st ACM international conference on Multimedia information retrieval, Vancouver, British Columbia, Canada.
• Li, X. Y., Shou, L. D., and Chen, G. 2006. A Latent Image Semantic Indexing Scheme for Image Retrieval on the Web. In International Conference of Web Information Systems, pp: 315-326.
• Popescu, A., Millet, C., and Moëllic, P.-A. 2007 .Ontology driven content based image retrieval. In Proceedings of the 6th ACM international conference on Image and video retrieval, Amsterdam, The Netherlands ,pp:387-394.
• Mezaris, V., Kompatsiaris, I., and Strintzis, M. G. 2003. An ontology approach to object-based image retrieval. In Proc. IEEE International Conference on Image Processing (ICIP 2003), Barcelona, Spain, Vol. 2, pp: 511-514.
• Shi, L., GU, G., Liu, H., and Shen, J. 2008. A Semantic Annotation Algorithm Based on Image Regional Object Ontology. In International Conference of Computer Science and Software Engineering, Vol. 4, pp: 540-543, Wuhan, Hubei.
• Ganea, E. 2010. An Object Oriented Graph Approach for Image Representation and Query Based on Content. In International Journal of Computer Science and Applications, Vol. 7 No. 1, pp: 45 – 59.
• Popescu, A., 2007. Image Retrieval Using a Multilingual Ontology. Proceeding RIAO '07 Large Scale Semantic Access to Content (Text, Image, Video, and Sound), pp: 461-474.
• Zhou, L. and Zhang, C. 2009. An Image Clustering and Retrieval Framework Using Feedback-based Integrated Region Matching. In International Conference on Machine Learning and Applications, pp: 422 – 427.
• Etzioni, O., Reiter, K., Soderland, S., and Sammer, M. 2007. Lexical translation with application to image search on the Web, in Proceedings of the in: Machine Translation Summit XI.