English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Recent transformer-based methods achieve notable gains in the Human-objectInteraction Detection (HOID) task by leveraging the detection of DETR and theprior knowledge of Vision-Language Model (VLM). However, these methods sufferfrom extended training times and complex optimization due to the entanglementof object detection and HOI recognition during the decoding process.Especially, the query embeddings used to predict both labels and boxes sufferfrom ambiguous representations, and the gap between the prediction of HOIlabels and verb labels is not considered. To address these challenges, weintroduce SOV-STG-VLA with three key components: Subject-Object-Verb (SOV)decoding, Specific Target Guided (STG) denoising, and a Vision-Language Advisor(VLA). Our SOV decoders disentangle object detection and verb recognition witha novel interaction region representation. The STG denoising strategy learnslabel embeddings with ground-truth information to guide the training andinference. Our SOV-STG achieves a fast convergence speed and high accuracy andbuilds a foundation for the VLA to incorporate the prior knowledge of the VLM.We introduce a vision advisor decoder to fuse both the interaction regioninformation and the VLM's vision knowledge and a Verb-HOI prediction bridge topromote interaction representation learning. Our VLA notably improves ourSOV-STG and achieves SOTA performance with one-sixth of training epochscompared to recent SOTA. Code and models are available athttps://github.com/cjw2021/SOV-STG-VLA

Focusing on what to decode and what to train: SOV Decoding with Specific Target Guided DeNoising and Vision Language Advisor