English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Background & Objective: Biomedical text data are increasingly available forresearch. Tokenization is an initial step in many biomedical text miningpipelines. Tokenization is the process of parsing an input biomedical sentence(represented as a digital character sequence) into a discrete set of word/tokensymbols, which convey focused semantic/syntactic meaning. The objective of thisstudy is to explore variation in tokenizer outputs when applied across a seriesof challenging biomedical sentences.  Method: Diaz [2015] introduce 24 challenging example biomedical sentences forcomparing tokenizer performance. In this study, we descriptively explorevariation in outputs of eight tokenizers applied to each example biomedicalsentence. The tokenizers compared in this study are the NLTK white spacetokenizer, the NLTK Penn Tree Bank tokenizer, Spacy and SciSpacy tokenizers,Stanza/Stanza-Craft tokenizers, the UDPipe tokenizer, and R-tokenizers.  Results: For many examples, tokenizers performed similarly effectively;however, for certain examples, there were meaningful variation in returnedoutputs. The white space tokenizer often performed differently than othertokenizers. We observed performance similarities for tokenizers implementingrule-based systems (e.g. pattern matching and regular expressions) andtokenizers implementing neural architectures for token classification.Oftentimes, the challenging tokens resulting in the greatest variation inoutputs, are those words which convey substantive and focusedbiomedical/clinical meaning (e.g. x-ray, IL-10, TCR/CD3, CD4+ CD8+, and(Ca2+)-regulated).  Conclusion: When state-of-the-art, open-source tokenizers from Python and Rwere applied to a series of challenging biomedical example sentences, weobserved subtle variation in the returned outputs.

Comparing Variation in Tokenizer Outputs Using a Series of Problematic and Challenging Biomedical Sentences