English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Image captioning is a task in the field of Artificial Intelligence thatmerges between computer vision and natural language processing. It isresponsible for generating legends that describe images, and has variousapplications like descriptions used by assistive technology or indexing images(for search engines for instance). This makes it a crucial topic in AI that isundergoing a lot of research. This task however, like many others, is trainedon large images labeled via human annotation, which can be very cumbersome: itneeds manual effort, both financial and temporal costs, it is error-prone andpotentially difficult to execute in some cases (e.g. medical images). Tomitigate the need for labels, we attempt to use self-supervised learning, atype of learning where models use the data contained within the imagesthemselves as labels. It is challenging to accomplish though, since the task istwo-fold: the images and captions come from two different modalities andusually handled by different types of networks. It is thus not obvious what acompletely self-supervised solution would look like. How it would achievecaptioning in a comparable way to how self-supervision is applied today onimage recognition tasks is still an ongoing research topic. In this project, weare using an encoder-decoder architecture where the encoder is a convolutionalneural network (CNN) trained on OpenImages dataset and learns image features ina self-supervised fashion using the rotation pretext task. The decoder is aLong Short-Term Memory (LSTM), and it is trained, along within the imagecaptioning model, on MS COCO dataset and is responsible of generating captions.Our GitHub repository can be found:https://github.com/elhagry1/SSL_ImageCaptioning_RotationPrediction

Experimenting with Self-Supervision using Rotation Prediction for Image Captioning