English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Event-based cameras offer a low-power alternative to frame-based cameras forcapturing high-speed motion and high dynamic range scenes. They provideasynchronous streams of sparse events. Spiking Neural Networks (SNNs) withtheir asynchronous event-driven compute, show great potential for extractingthe spatio-temporal features from these event streams. In contrast, thestandard Analog Neural Networks (ANNs1) fail to process event data effectively.However, training SNNs is difficult due to additional trainable parameters(thresholds and leaks), vanishing spikes at deeper layers, non-differentiablebinary activation function etc. Moreover, an additional data structure"membrane potential" responsible for keeping track of temporal information,must be fetched and updated at every timestep in SNNs. To overcome these, wepropose a novel SNN-ANN hybrid architecture that combines the strengths ofboth. Specifically, we leverage the asynchronous compute capabilities of SNNlayers to effectively extract the input temporal information. While the ANNlayers offer trouble-free training and implementation on standard machinelearning hardware such as GPUs. We provide extensive experimental analysis forassigning each layer to be spiking or analog in nature, leading to a networkconfiguration optimized for performance and ease of training. We evaluate ourhybrid architectures for optical flow estimation using event-data on DSEC-flowand Mutli-Vehicle Stereo Event-Camera (MVSEC) datasets. The results indicatethat our configured hybrid architectures outperform the state-of-the-artANN-only, SNN-only and past hybrid architectures both in terms of accuracy andefficiency. Specifically, our hybrid architecture exhibit a 31% and 24.8% loweraverage endpoint error (AEE) at 2.1x and 3.1x lower energy, compared to anSNN-only architecture on DSEC and MVSEC datasets, respectively.

Best of Both Worlds: Hybrid SNN-ANN Architecture for Event-based Optical Flow Estimation