Distributed Multimodal 2.4 GHz Wi-Fi Received Signal Strength Indicator Prediction for Wireless Resilient Robot Operation

This study aims to build a practical deep learning-based framework for resilient robot communication. We propose a proactive robot controller for each local robot to ensure a stable received signal by predicting future signal quality. The predictor is trained using multiple types of data experimentally obtained in robot operation up to the present, which include camera images, point clouds, and 2.4 GHzWi-Fi channel state information using multimodal neural networks for local robots. The framework faces practical challenges in real-world deployments, such as synchronizing heterogeneous data modalities, aligning data with different spatial and temporal resolutions, and ensuring accuracy, performance, and robustness under dynamic environmental conditions typical of mobile robotics scenarios. The results show that the proposed multimodal predictor outperforms single-modality approaches in terms of root-mean-squared error (2.34 dB), accuracy (79%), correlation coefficient (0.88), and coefficient of determination (0.76). A comparison of the accuracy and performance of our proposed customized multimodal three-dimensional residual network (CM-3DResNet) and state-of-the-art architectures of 3DResNet-18, 3DResNet-50, and Vision Transformer shows feasibility in future link quality prediction tasks for robots. Aggregating the local results, a federated learning-based framework with a prediction result of 2.80 dB for two robots and 3.14 dB for three robots in root-mean-squared error compared to the ground truth is proposed as a candidate for cooperative robots operating in severe wireless environments at 2.4 GHz. The prediction results from the proposed federated learning paradigm surpassed those of Learning without Forgetting approach in terms of multitask prediction.