English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Pre-trained Transformers are good foundations for unified multi-task modelsowing to their task-agnostic representation. Pre-trained Transformers are oftencombined with text-to-text framework to execute multiple tasks by a singlemodel. Performing a task through a graphical user interface (GUI) is anothercandidate to accommodate various tasks, including multi-step tasks with visionand language inputs. However, few papers combine pre-trained Transformers withperforming through GUI. To fill this gap, we explore a framework in which amodel performs a task by manipulating the GUI implemented with web pages inmultiple steps. We develop task pages with and without page transitions andpropose a BERT extension for the framework. We jointly trained our BERTextension with those task pages, and made the following observations. (1) Themodel learned to use both task pages with and without page transition. (2) Infour out of five tasks without page transitions, the model performs greaterthan 75% of the performance of the original BERT, which does not use browsers.(3) The model did not generalize effectively on unseen tasks. These resultssuggest that we can fine-tune BERTs to multi-step tasks through GUIs, and thereis room for improvement in their generalizability. Code will be availableonline.

Do BERTs Learn to Use Browser User Interface? Exploring Multi-Step Tasks with Unified Vision-and-Language BERTs