English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools annual

Global: Zendy Free Plan

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Increasing model, data and compute budget scale in the pre-training has beenshown to strongly improve model generalization and transfer learning in vastline of work done in language modeling and natural image recognition. However,most studies on the positive effect of larger scale were done in scope ofin-domain setting, with source and target data being in close proximity. Tostudy effect of larger scale for both in-domain and out-of-domain setting whenperforming full and few-shot transfer, we combine here for the first timelarge, openly available medical X-Ray chest imaging datasets to reach a scalefor medical imaging domain comparable to ImageNet-1k, routinely used forpre-training in natural image domain. We then conduct supervised pre-training,while varying network size and source data scale and domain, being either largenatural (ImageNet-1k/21k) or large medical chest X-Ray datasets, and transferpre-trained models to different natural or medical targets. We observe strongimprovement due to larger pre-training scale for intra-domain natural-naturaland medical-medical transfer. For inter-domain natural-medical transfer, wefind improvements due to larger pre-training scale on larger X-Ray targets infull shot regime, while for smaller targets and for few-shot regime theimprovement is not visible. Remarkably, large networks pre-trained on verylarge natural ImageNet-21k are as good or better than networks pre-trained onlargest available medical X-Ray data when performing transfer to large X-Raytargets. We conclude that substantially increasing model and generic, medicaldomain-agnostic natural image source data scale in the pre-training can enablehigh quality out-of-domain transfer to medical domain specific targets,removing dependency on large medical domain-specific source data often notavailable in the practice.

Effect of pre-training scale on intra- and inter-domain, full and few-shot transfer learning for natural and X-Ray chest images