AUTOMATIC CAPTION GENERATION FOR NEWS IMAGES USING EXTRACTIVE AND ABSTRACTIVE MODELS

Abstract

Automatic image caption generation is of great interest to many image related applications. Now a day’s, whenever retrieving
images from the search Engines that retrieves images without analyzing their content, simply by matching user queries against the
image’s file name and format, user-annotated tags, captions, and, generally, text surrounding the image. Also the retrieved image
does not contain any textual data along with the images. We introduced the task of automatic caption generation for news images.
The task fuses insights from computer vision and natural language processing and holds promise for various multimedia
applications, such as image retrieval, development of tools supporting news media management, and for individuals with visual
impairment. It is possible to learn a caption generation model from weakly labelled data without costly manual involvement.
Instead of manually creating annotations, image captions are treated as labels for the image. Although the caption words are
admittedly noisy compared to traditional human-created keywords, we show that they can be used to learn the correspondences
between visual and textual modalities, and also serve as a gold standard for the caption generation task. We have presented
extractive and abstractive caption generation models. A key aspect of our approach is to allow both the visual and textual
modalities to influence the generation task.