English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Controllable Image Captioning (CIC) -- generating image descriptionsfollowing designated control signals -- has received unprecedented attentionover the last few years. To emulate the human ability in controlling captiongeneration, current CIC studies focus exclusively on control signals concerningobjective properties, such as contents of interest or descriptive patterns.However, we argue that almost all existing objective control signals haveoverlooked two indispensable characteristics of an ideal control signal: 1)Event-compatible: all visual contents referred to in a single sentence shouldbe compatible with the described activity. 2) Sample-suitable: the controlsignals should be suitable for a specific image sample. To this end, we proposea new control signal for CIC: Verb-specific Semantic Roles (VSR). VSR consistsof a verb and some semantic roles, which represents a targeted activity and theroles of entities involved in this activity. Given a designated VSR, we firsttrain a grounded semantic role labeling (GSRL) model to identify and ground allentities for each role. Then, we propose a semantic structure planner (SSP) tolearn human-like descriptive semantic structures. Lastly, we use a role-shiftcaptioning model to generate the captions. Extensive experiments and ablationsdemonstrate that our framework can achieve better controllability than severalstrong baselines on two challenging CIC benchmarks. Besides, we can generatemulti-level diverse captions easily. The code is available at:https://github.com/mad-red/VSR-guided-CIC.

Human-like Controllable Image Captioning with Verb-specific Semantic Roles