English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The recent success of ChatGPT and GPT-4 has drawn widespread attention tomultimodal dialogue systems. However, the academia community lacks a datasetthat can validate the multimodal generation capabilities of Visual LanguageModels (VLMs) in textual-visual chat tasks. In this paper, we construct two newmultimodal datasets: the synthetic CLEVR-ATVC dataset (620K) and the manuallypictured Fruit-ATVC dataset (50K), both featuring visual and text-based inputsand outputs. Additionally, to enable the multimodal system to reject humanrequests (i.e., demonstrate accountability), as in language-based ChatGPTconversations, we develop and incorporate specific rules into the datasets assupervisory signals. This allows the trained VLM to provide a yes or no answerafter visual and textual reasoning, accompanied by a language explanation as towhy the human instruction cannot be excuted. In our method, we propose atwo-state training procedure to train the image auto-encoder andauto-regressive transformer from scratch. The first state involves a discretevariational autoencoder (dVAE) to compress each image into short tokens, whichare then concatenated with text tokens as a single data stream to be fed intothe decoder-based transformer for generating visual re-creation and textualfeedback in the second state. We provide comprehensive analyses of experimentalresults in terms of re-created image quality, answer accuracy, and the modelbehavior when faced with uncertainty and imperfect user queries. We hope ourexplorations and findings contribute valuable insights regarding theaccountability of textual-visual generative models.

Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation