Multimodal Issues and Key Technologies in Ocean Observation Data: a Review

Ocean observation data exhibits multimodal characteristics due to its diverse sources, heterogeneous data structures, and varied parameters. Integrating and correlating these data sources is a critical technology for enhancing comprehensive oceanic understanding. Unlike traditional multimodal problems in unstructured data, ocean observation data are structured scientific data, posing unique challenges. This paper is the first to identify and define three core multimodal issues in ocean observation data: multi-structure, multi-element, and multi-source. We systematically review recent studies leveraging multimodal ocean observation data, summarizing and discussing these works from the perspectives of application scenarios and technical methodologies. Particular focus is given to cutting-edge technologies such as deep learning, knowledge graphs, federated learning, and physics-informed neural networks. Finally, we highlight the limitations of existing methods in addressing multimodal issues in ocean observation data and outline promising directions for future research. This comprehensive and systematic review aims to serve as a valuable reference for researchers in ocean science and data processing domains.