English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Language models, primarily transformer-based ones, obtained colossal successin NLP. To be more precise, studies like BERT in NLU and works such as GPT-3for NLG are very crucial. DNA sequences are very close to natural language interms of structure, so if the DNA-related bioinformatics domain is concerned,discriminative models, like DNABert, exist. Yet, the generative side of thecoin is mainly unexplored to the best of our knowledge. Consequently, wefocused on developing an autoregressive generative language model like GPT-3for DNA sequences. Because working with whole DNA sequences is challengingwithout substantial computational resources, we decided to carry out our studyon a smaller scale, focusing on nucleotide sequences of human genes, uniqueparts in DNA with specific functionalities, instead of the whole DNA. Thisdecision did not change the problem structure a lot due to the fact that bothDNA and genes can be seen as 1D sequences consisting of four differentnucleotides without losing much information and making too much simplification.First of all, we systematically examined an almost entirely unexplored problemand observed that RNNs performed the best while simple techniques like N-gramswere also promising. Another beneficial point was learning how to work withgenerative models on languages we do not understand, unlike natural language.How essential using real-life tasks beyond the classical metrics such asperplexity is observed. Furthermore, checking whether the data-hungry nature ofthese models can be changed through selecting a language with minimalvocabulary size, four owing to four different types of nucleotides, isexamined. The reason for reviewing this was that choosing such a language mightmake the problem easier. However, what we observed in this study was it did notprovide that much of a change in the amount of data needed.

Generative Language Models on Nucleotide Sequences of Human Genes