English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Biological sensing and processing is asynchronous and sparse, leading tolow-latency and energy-efficient perception and action. In robotics,neuromorphic hardware for event-based vision and spiking neural networkspromises to exhibit similar characteristics. However, robotic implementationshave been limited to basic tasks with low-dimensional sensory inputs and motoractions due to the restricted network size in current embedded neuromorphicprocessors and the difficulties of training spiking neural networks. Here, wepresent the first fully neuromorphic vision-to-control pipeline for controllinga freely flying drone. Specifically, we train a spiking neural network thataccepts high-dimensional raw event-based camera data and outputs low-levelcontrol actions for performing autonomous vision-based flight. The vision partof the network, consisting of five layers and 28.8k neurons, maps incoming rawevents to ego-motion estimates and is trained with self-supervised learning onreal event data. The control part consists of a single decoding layer and islearned with an evolutionary algorithm in a drone simulator. Roboticexperiments show a successful sim-to-real transfer of the fully learnedneuromorphic pipeline. The drone can accurately follow different ego-motionsetpoints, allowing for hovering, landing, and maneuveringsideways$\unicode{x2014}$even while yawing at the same time. The neuromorphicpipeline runs on board on Intel's Loihi neuromorphic processor with anexecution frequency of 200 Hz, spending only 27 $\unicode{x00b5}$J perinference. These results illustrate the potential of neuromorphic sensing andprocessing for enabling smaller, more intelligent robots.

Fully neuromorphic vision and control for autonomous drone flight