English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Story visualization is an under-explored task that falls at the intersectionof many important research directions in both computer vision and naturallanguage processing. In this task, given a series of natural language captionswhich compose a story, an agent must generate a sequence of images thatcorrespond to the captions. Prior work has introduced recurrent generativemodels which outperform text-to-image synthesis models on this task. However,there is room for improvement of generated images in terms of visual quality,coherence and relevance. We present a number of improvements to prior modelingapproaches, including (1) the addition of a dual learning framework thatutilizes video captioning to reinforce the semantic alignment between the storyand generated images, (2) a copy-transform mechanism forsequentially-consistent story visualization, and (3) MART-based transformers tomodel complex interactions between frames. We present ablation studies todemonstrate the effect of each of these techniques on the generative power ofthe model for both individual images as well as the entire narrative.Furthermore, due to the complexity and generative nature of the task, standardevaluation metrics do not accurately reflect performance. Therefore, we alsoprovide an exploration of evaluation metrics for the model, focused on aspectsof the generated frames such as the presence/quality of generated characters,the relevance to captions, and the diversity of the generated images. We alsopresent correlation experiments of our proposed automated metrics with humanevaluations. Code and data available at:https://github.com/adymaharana/StoryViz

Improving Generation and Evaluation of Visual Stories via Semantic Consistency