Human inspired deep learning to locate and classify terrestrial and arboreal animals in thermal drone surveys

Abstract Drones are an effective tool for animal surveys, capable of generating an abundance of high‐quality ecological data. However, the large volume of ecological data generated introduces an additional problem of the requisite human resources to process and analyse such data. Deep learning models offer a solution to this challenge, capable of autonomously processing drone footage to detect animals with higher fidelity and lower latency when compared with humans. This work aimed to develop an animal detection architecture that classifies animals in accordance to their location (terrestrial vs. arboreal). The model incorporates human pilot inspired techniques for greater performance and consistency across time. Thermal drone footage across the state of New South Wales, Australia from surveys over a 2+ year period was used to construct a diverse training and validation dataset. A high‐resolution 3D simulation was developed to workload by autonomously generating labelled data to supplement manually labelled field data. The model was evaluated on 130 hours of thermal imagery (14 million images) containing 57 unique animal species where 1637 out of 1719 (95.23%) of human pilot recorded animals were detected. The model achieved an F1 score of 0.9410, a 4.36 percentage point increase in performance over a benchmark YOLOv8 model. Simulated data improved model performance by 1.7x for low data scenarios, lowering data labelling costs due to higher quality image pre‐labels. The proposed animal detection model demonstrates strong reporting accuracy in the detection and tracking of animals. The approach enables widespread adoption of drone‐capturing technology by providing in‐field real‐time assistance, allowing novice pilots to detect animals at the level of experienced pilots, whilst also reducing the burden of report generation and data labelling costs.