Efficient pavement distress classification via deep patch soft selective learning and knowledge distillation

Pavement distress classification is a vital step for automatic pavement inspection and maintenance. Recently, patch‐based approaches have achieved promising performances and thus extensive attention in this field. However, these methods simply assume that all patches contribute equally to the distress classification, leading to weakly discriminating abilities of models. Moreover, their tedious processes also lead to low efficiency in inference. The authors present a novel patch‐based pavement distress classification approach named Deep Patch Soft Selective Learning, which addresses these issues. Similar to other patch‐based approaches, Deep Patch Soft Selective Learning partitions the pavement images into patches and aggregates the patch features to accomplish the task. To address the first issue, a succinct Soft Patch Feature Selection Network is introduced to assess the importance of each patch to the distress classification with a score based on its feature. These scores will be considered as patch‐wise weights for feature aggregation. In such a manner, the most discriminative patches are selected in a soft way, and thereby benefit the final classification. To address the inference efficiency issue, knowledge distillation is leveraged to transfer the classification knowledge from Deep Patch Soft Selective Learning to the image‐based approaches, such as EfficientNet‐B3. This distilled model enables incorporating both the advantages of patch‐based approaches in classification performance and the advantages of image‐based approaches in inference efficiency. Extensive experiments on a large‐scale pavement image dataset named CQU‐BPDD demonstrate that Deep Patch Soft Selective Learning outperforms the baselines by a large margin of +7.4% in P@R on the detection task and +3.1% in F 1 on the recognition task while enjoying 2.1x throughput of baselines.