English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent developments of multi-modal large language models have demonstratedits strong ability in solving vision-language tasks. In this paper, we focus onthe product understanding task, which plays an essential role in enhancingonline shopping experience. Product understanding task includes a variety ofsub-tasks, which require models to respond diverse queries based on multi-modalproduct information. Traditional methods design distinct model architecturesfor each sub-task. On the contrary, we present PUMGPT, a large vision-languagemodel aims at unifying all product understanding tasks under a singular modelstructure. To bridge the gap between vision and text representations, wepropose Layer-wise Adapters (LA), an approach that provides enhanced alignmentwith fewer visual tokens and enables parameter-efficient fine-tuning. Moreover,the inherent parameter-efficient fine-tuning ability allows PUMGPT to bereadily adapted to new product understanding tasks and emerging products. Wedesign instruction templates to generate diverse product instruction datasets.Simultaneously, we utilize open-domain datasets during training to improve theperformance of PUMGPT and its generalization ability. Through extensiveevaluations, PUMGPT demonstrates its superior performance across multipleproduct understanding tasks, including product captioning, categoryquestion-answering, attribute extraction, attribute question-answering, andeven free-form question-answering about products.

PUMGPT: A Large Vision-Language Model for Product Understanding