English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

In this paper we delve into the properties of transformers, attained throughself-supervision, in the point cloud domain. Specifically, we evaluate theeffectiveness of Masked Autoencoding as a pretraining scheme, and exploreMomentum Contrast as an alternative. In our study we investigate the impact ofdata quantity on the learned features, and uncover similarities in thetransformer's behavior across domains. Through comprehensive visualiations, weobserve that the transformer learns to attend to semantically meaningfulregions, indicating that pretraining leads to a better understanding of theunderlying geometry. Moreover, we examine the finetuning process and its effecton the learned representations. Based on that, we devise an unfreezing strategywhich consistently outperforms our baseline without introducing any othermodifications to the model or the training pipeline, and achievestate-of-the-art results in the classification task among transformer models.

ExpPoint-MAE: Better interpretability and performance for self-supervised point cloud transformers