English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

In this paper, we determine analytical bounds on the local Lipschitzconstants of of affine functions composed with rectified linear units (ReLUs).Affine-ReLU functions represent a widely used layer in deep neural networks,due to the fact that convolution, fully-connected, and normalization functionsare all affine, and are often followed by a ReLU activation function. Using ananalytical approach, we mathematically determine upper bounds on the localLipschitz constant of an affine-ReLU function, show how these bounds can becombined to determine a bound on an entire network, and discuss how the boundscan be efficiently computed, even for larger layers and networks. We showseveral examples by applying our results to AlexNet, as well as several smallernetworks based on the MNIST and CIFAR-10 datasets. The results show that ourmethod produces tighter bounds than the standard conservative bound (i.e. theproduct of the spectral norms of the layers' linear matrices), especially forsmall perturbations.

Analytical bounds on the local Lipschitz constants of affine-ReLU functions