Key Variables in the Reliability of ML Models Exposed to Neutrons, Protons, and Heavy Ions

Machine learning (ML) models are able to process complex images, providing state-of-the-art performance in tasks such as image classification and semantic segmentation. These models can be mapped to highly efficient commercial-off-the-shelf (COTS) specialized hardware accelerators, whose reliability should be carefully evaluated before deployment. Unfortunately, given the large number of ML model architectures, possible configurations or input selections, and the numerous COTS accelerator architectures available, exhaustively testing every model-accelerator combination with beam experiments is unfeasible. Additionally, the radiation data obtained with a specific combination can hardly be extended to different configurations. In this paper, we test vision transformer (ViT) and segmentation convolutional neural network (CNN) models, in addition to several ML micro-benchmarks, on the Google Coral Edge TPU at 6 different radiation facilities, investigating particle-, software-, and hardware-dependent reliability behaviors. Our experimental results show that, while the cross section for radiation-induced silent data corruption (SDCs) can be up to 8 orders-of-magnitude higher when testing with high-LET heavy ions compared to atmospheric neutrons, the characteristics of the SDCs are similar across all types of radiation tested. Instead, the most impactful factors that lead to misclassifications in beam tests are actually the model complexity and the input selection . These results can be leveraged to more efficiently plan and utilize the beam time available in radiation experiments, thus improving the understanding of the fault models affecting the software, while also characterizing the reliability of the underlying hardware accelerator.