English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recent works indicate that convolutional neural networks (CNN) need largereceptive fields (RF) to compete with visual transformers and their attentionmechanism. In CNNs, RFs can simply be enlarged by increasing the convolutionkernel sizes. Yet the number of trainable parameters, which scalesquadratically with the kernel's size in the 2D case, rapidly becomesprohibitive, and the training is notoriously difficult. This paper presents anew method to increase the RF size without increasing the number of parameters.The dilated convolution (DC) has already been proposed for the same purpose. DCcan be seen as a convolution with a kernel that contains only a few non-zeroelements placed on a regular grid. Here we present a new version of the DC inwhich the spacings between the non-zero elements, or equivalently theirpositions, are no longer fixed but learnable via backpropagation thanks to aninterpolation technique. We call this method "Dilated Convolution withLearnable Spacings" (DCLS) and generalize it to the n-dimensional convolutioncase. However, our main focus here will be on the 2D case. We first tried ourapproach on ResNet50: we drop-in replaced the standard convolutions with DCLSones, which increased the accuracy of ImageNet1k classification atiso-parameters, but at the expense of the throughput. Next, we used the recentConvNeXt state-of-the-art convolutional architecture and drop-in replaced thedepthwise convolutions with DCLS ones. This not only increased the accuracy ofImageNet1k classification but also of typical downstream and robustness tasks,again at iso-parameters but this time with negligible cost on throughput, asConvNeXt uses separable convolutions. Conversely, classic DC led to poorperformance with both ResNet50 and ConvNeXt. The code of the method isavailable at:https://github.com/K-H-Ismail/Dilated-Convolution-with-Learnable-Spacings-PyTorch.

Dilated convolution with learnable spacings