Please use this identifier to cite or link to this item:
https://idr.l3.nitk.ac.in/jspui/handle/123456789/7117
Title: | A robust approach to open vocabulary image retrieval with deep convolutional neural networks and transfer learning |
Authors: | Padmakumar, V. Ranga, R. Elluru, S. Sowmya, Kamath S. |
Issue Date: | 2018 |
Citation: | Proceedings of the 2018 Pacific Neighborhood Consortium Annual Conference and Joint Meetings: Human Rights in Cyberspace, PNC 2018, 2018, Vol., , pp.106-112 |
Abstract: | Enabling computer systems to respond to conversational human language is a challenging problem with wideranging applications in the field of robotics and human computer interaction. Specifically, in image searches, humans tend to describe objects in fine-grained detail like color or company, for which conventional retrieval algorithms have shown poor performance. In this paper, a novel approach for open vocabulary image retrieval, capable of selecting the correct candidate image from among a set of distractions given a query in natural language form, is presented. Our methodology focuses on generating a robust set of image-text projections capable of accurately representing any image, with an objective of achieving high recall. To this end, an ensemble of classifiers is trained on ImageNet for representing high-resolution objects, Cifar 100 for smaller resolution images of objects and Caltech 256 for challenging views of everyday objects, for generating category-based projections. In addition to category based projections, we also make use of an image captioning model trained on MS COCO and Google Image Search (GISS) to capture additional semantic/latent information about the candidate images. To facilitate image retrieval, the natural language query and projection results are converted to a common vector representation using word embeddings, with which query-image similarity is computed. The proposed model when benchmarked on the RefCoco dataset, achieved an accuracy of 68.8%, while retrieving semantically meaningful candidate images. � 2018 Pacific Neighborhood Consortium (PNC). |
URI: | https://idr.nitk.ac.in/jspui/handle/123456789/7117 |
Appears in Collections: | 2. Conference Papers |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
9 A Robust Approach.pdf | 2.17 MB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.