English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

With the emergence of Artificial Intelligence (AI), new attention has beengiven to implement AI algorithms on resource constrained tiny devices to expandthe application domain of IoT. Multimodal Learning has recently become verypopular with the classification task due to its impressive performance for bothimage and audio event classification. This paper presents TinyM$^2$Net -- aflexible system algorithm co-designed multimodal learning framework forresource constrained tiny devices. The framework was designed to be evaluatedon two different case-studies: COVID-19 detection from multimodal audiorecordings and battle field object detection from multimodal images and audios.In order to compress the model to implement on tiny devices, substantialnetwork architecture optimization and mixed precision quantization wereperformed (mixed 8-bit and 4-bit). TinyM$^2$Net shows that even a tinymultimodal learning model can improve the classification performance than thatof any unimodal frameworks. The most compressed TinyM$^2$Net achieves 88.4%COVID-19 detection accuracy (14.5% improvement from unimodal base model) and96.8% battle field object detection accuracy (3.9% improvement from unimodalbase model). Finally, we test our TinyM$^2$Net models on a Raspberry Pi 4 tosee how they perform when deployed to a resource constrained tiny device.

TinyM$^2$Net: A Flexible System Algorithm Co-designed Multimodal Learning Framework for Tiny Devices