English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Modern image-to-text systems typically adopt the encoder-decoder framework,which comprises two main components: an image encoder, responsible forextracting image features, and a transformer-based decoder, used for generatingcaptions. Taking inspiration from the analysis of neural networks' robustnessagainst adversarial perturbations, we propose a novel gray-box algorithm forcreating adversarial examples in image-to-text models. Unlike imageclassification tasks that have a finite set of class labels, finding visuallysimilar adversarial examples in an image-to-text task poses greater challengesbecause the captioning system allows for a virtually infinite space of possiblecaptions. In this paper, we present a gray-box adversarial attack onimage-to-text, both untargeted and targeted. We formulate the process ofdiscovering adversarial perturbations as an optimization problem that uses onlythe image-encoder component, meaning the proposed attack is language-modelagnostic. Through experiments conducted on the ViT-GPT2 model, which is themost-used image-to-text model in Hugging Face, and the Flickr30k dataset, wedemonstrate that our proposed attack successfully generates visually similaradversarial examples, both with untargeted and targeted captions. Notably, ourattack operates in a gray-box manner, requiring no knowledge about the decodermodule. We also show that our attacks fool the popular open-source platformHugging Face.

I See Dead People: Gray-Box Adversarial Attack on Image-To-Text Models