English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Contextualized representations from a pre-trained language model are centralto achieve a high performance on downstream NLP task. The pre-trained BERT andA Lite BERT (ALBERT) models can be fine-tuned to give state-ofthe-art resultsin sentence-pair regressions such as semantic textual similarity (STS) andnatural language inference (NLI). Although BERT-based models yield the [CLS]token vector as a reasonable sentence embedding, the search for an optimalsentence embedding scheme remains an active research area in computationallinguistics. This paper explores on sentence embedding models for BERT andALBERT. In particular, we take a modified BERT network with siamese and tripletnetwork structures called Sentence-BERT (SBERT) and replace BERT with ALBERT tocreate Sentence-ALBERT (SALBERT). We also experiment with an outer CNNsentence-embedding network for SBERT and SALBERT. We evaluate performances ofall sentence-embedding models considered using the STS and NLI datasets. Theempirical results indicate that our CNN architecture improves ALBERT modelssubstantially more than BERT models for STS benchmark. Despite significantlyfewer model parameters, ALBERT sentence embedding is highly competitive to BERTin downstream NLP evaluations.

Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks