English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Generating dialogue grounded in videos requires a high level of understandingand reasoning about the visual scenes in the videos. However, existing largevisual-language models are not effective due to their latent features anddecoder-only structure, especially with respect to spatio-temporal relationshipreasoning. In this paper, we propose a novel approach named MSG-BART, whichenhances the integration of video information by incorporating amulti-granularity spatio-temporal scene graph into an encoder-decoderpre-trained language model. Specifically, we integrate the global and localscene graph into the encoder and decoder, respectively, to improve both overallperception and target reasoning capability. To further improve the informationselection capability, we propose a multi-pointer network to facilitateselection between text and video. Extensive experiments are conducted on threevideo-grounded dialogue benchmarks, which show the significant superiority ofthe proposed MSG-BART compared to a range of state-of-the-art approaches.

MSG-BART: Multi-granularity Scene Graph-Enhanced Encoder-Decoder Language Model for Video-grounded Dialogue Generation