English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent advances in vision-language pre-trained models (VLPs) havesignificantly increased visual understanding and cross-modal analysiscapabilities. Companies have emerged to provide multi-modal Embedding as aService (EaaS) based on VLPs (e.g., CLIP-based VLPs), which cost a large amountof training data and resources for high-performance service. However, existingstudies indicate that EaaS is vulnerable to model extraction attacks thatinduce great loss for the owners of VLPs. Protecting the intellectual propertyand commercial ownership of VLPs is increasingly crucial yet challenging. Amajor solution of watermarking model for EaaS implants a backdoor in the modelby inserting verifiable trigger embeddings into texts, but it is onlyapplicable for large language models and is unrealistic due to data and modelprivacy. In this paper, we propose a safe and robust backdoor-based embeddingwatermarking method for VLPs called VLPMarker. VLPMarker utilizes embeddingorthogonal transformation to effectively inject triggers into the VLPs withoutinterfering with the model parameters, which achieves high-quality copyrightverification and minimal impact on model performance. To enhance the watermarkrobustness, we further propose a collaborative copyright verification strategybased on both backdoor trigger and embedding distribution, enhancing resilienceagainst various attacks. We increase the watermark practicality via anout-of-distribution trigger selection approach, removing access to the modeltraining data and thus making it possible for many real-world scenarios. Ourextensive experiments on various datasets indicate that the proposedwatermarking approach is effective and safe for verifying the copyright of VLPsfor multi-modal EaaS and robust against model extraction attacks. Our code isavailable at https://github.com/Pter61/vlpmarker.

Watermarking Vision-Language Pre-trained Models for Multi-modal Embedding as a Service