English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Large vision-language models (VLMs) such as GPT-4 have achieved exceptionalperformance across various multi-modal tasks. However, the deployment of VLMsnecessitates substantial energy consumption and computational resources. Onceattackers maliciously induce high energy consumption and latency time(energy-latency cost) during inference of VLMs, it will exhaust computationalresources. In this paper, we explore this attack surface about availability ofVLMs and aim to induce high energy-latency cost during inference of VLMs. Wefind that high energy-latency cost during inference of VLMs can be manipulatedby maximizing the length of generated sequences. To this end, we proposeverbose images, with the goal of crafting an imperceptible perturbation toinduce VLMs to generate long sentences during inference. Concretely, we designthree loss objectives. First, a loss is proposed to delay the occurrence ofend-of-sequence (EOS) token, where EOS token is a signal for VLMs to stopgenerating further tokens. Moreover, an uncertainty loss and a token diversityloss are proposed to increase the uncertainty over each generated token and thediversity among all tokens of the whole generated sequence, respectively, whichcan break output dependency at token-level and sequence-level. Furthermore, atemporal weight adjustment algorithm is proposed, which can effectively balancethese losses. Extensive experiments demonstrate that our verbose images canincrease the length of generated sequences by 7.87 times and 8.56 timescompared to original images on MS-COCO and ImageNet datasets, which presentspotential challenges for various applications. Our code is available athttps://github.com/KuofengGao/Verbose_Images.

Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images