Multimodal Information Extraction from Visually Rich Documents with Adaptive Graph Integration Network

In the context of factory informatization, there is a growing need to automatically obtain information from diverse document images. Multimodal information extraction from visually rich documents (VRDs) remains challenging due to complex text layouts, local noise, and uneven text distribution. Methods such as LayoutLM obtain strong results but incur high computational costs and are not designed to be lightweight. Although GraphReviseIE is relatively lightweight, it still has limitations—for example, it insufficiently exploits spatial relational features.To address this issue, we propose an Adaptive Graph Integration Network (AGIN). Our model introduces a novel 2D spatial relative-orientation positional embedding that more effectively captures spatial relationships among text segments, enriching multimodal representations. We also design a dual-graph adaptive integration mechanism composed of (1) a self-revised graph that captures implicit structural dependencies via node similarity propagation, and (2) a relation-revised graph that encodes explicit semantic-spatial relations between text segments. These graphs are adaptively consolidated using learnable attention weights, which mitigates the stochastic variations commonly observed in traditional graph constructions. Additionally, we adopt a multitask learning framework. Experiments on multiple real-world datasets show that our approach improves information-extraction capability and achieves comparable or superior performance relative to baseline methods.