English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Tools annual

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Tools monthly

Global: Zendy Plus monthly

Presents the access of premium content as premium feature

Premium Content

Global: Zendy Plus annual

Global: Zendy Free Plan

Less than 1% of protein sequences are structurally and functionallyannotated. Natural Language Processing (NLP) community has recently embracedself-supervised learning as a powerful approach to learn representations fromunlabeled text, in large part due to the attention-based context-awareTransformer models. In this work we present a modification to the RoBERTa modelby inputting during pre-training a mixture of binding and non-binding proteinsequences (from STRING database). However, the sequence pairs have no label toindicate their binding status, as the model relies solely on Masked LanguageModeling (MLM) objective during pre-training. After fine-tuning, such approachsurpasses models trained on single protein sequences for protein-proteinbinding prediction, TCR-epitope binding prediction, cellular-localization andremote homology classification tasks. We suggest that the Transformer'sattention mechanism contributes to protein binding site discovery. Furthermore,we compress protein sequences by 64% with the Byte Pair Encoding (BPE)vocabulary consisting of 10K subwords, each around 3-4 amino acids long.Finally, to expand the model input space to even larger proteins andmulti-protein assemblies, we pre-train Longformer models that support 2,048tokens. Further work in token-level classification for secondary structureprediction is needed. Code available at:https://github.com/PaccMann/paccmann_proteomics

Pre-training Protein Language Models with Label-Agnostic Binding Pairs Enhances Performance in Downstream Tasks