Open AccessIn-Context Vision-Pattern-Language Model for Enhancing Vessel Activity Explanation
Author(s) -
Changha Lee,
Chan-Hyun Youn
Publication year - 2025
Publication title -
ieee access
Language(s) - English
Resource type - Magazines
SCImago Journal Rank - 0.587
H-Index - 127
eISSN - 2169-3536
DOI - 10.1109/access.2025.3572714
Subject(s) - aerospace , bioengineering , communication, networking and broadcast technologies , components, circuits, devices and systems , computing and processing , engineered materials, dielectrics and plasmas , engineering profession , fields, waves and electromagnetics , general topics for engineers , geoscience , nuclear engineering , photonics and electrooptics , power, energy and industry applications , robotics and control systems , signal processing and analysis , transportation
Illegal vessel activities pose significant threats to marine resources and ecosystems worldwide, necessitating effective monitoring and detection methods. Current vessel monitoring systems struggle to accurately interpret vessel behaviors at a detailed level due to limitations in utilizing multi-modal data and regulatory frameworks. To overcome these challenges, we propose a new Vision-Pattern-Language (VPL) model designed to enhance the explanation and detection of illegal, unreported, and unregulated (IUU) vessel activities. To handle AIS-off in boundary waters, proposed model integrates satellite imagery with Automatic Identification System (AIS) trajectory data using a probabilistic fusion approach based on Maximum A Posteriori (MAP) estimation with Monte Carlo dropout. Additionally, the VPL model employs a CLIP-based zero-shot classifier to accurately identify vessel behaviors. To support law enforcement, the VPL model with in-context learning also generates faithful and contextually reasonable explanations grounded in the fused data and a legal-text database. Extensive experimental evaluations on the AIS and satellite imagery dataset demonstrate that the VPL model significantly improves trajectory prediction accuracy and classification performance than baselines. Moreover, VPL attains higher faithfulness and reasoning scores compared to Llama-3.3, highlighting its potential for robust and reliable maritime surveillance and contributing meaningfully to the detection and regulation of IUU vessel activities.