English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Zendy Plus

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Zendy Tools

Zendy Open

Heavily pre-trained transformers for language modelling, such as BERT, haveshown to be remarkably effective for Information Retrieval (IR) tasks,typically applied to re-rank the results of a first-stage retrieval model. IRbenchmarks evaluate the effectiveness of retrieval pipelines based on thepremise that a single query is used to instantiate the underlying informationneed. However, previous research has shown that (I) queries generated by usersfor a fixed information need are extremely variable and, in particular, (II)neural models are brittle and often make mistakes when tested with modifiedinputs. Motivated by those observations we aim to answer the followingquestion: how robust are retrieval pipelines with respect to differentvariations in queries that do not change the queries' semantics? In order toobtain queries that are representative of users' querying variability, we firstcreated a taxonomy based on the manual annotation of transformations occurringin a dataset (UQV100) of user-created query variations. For eachsyntax-changing category of our taxonomy, we employed different automaticmethods that when applied to a query generate a query variation. Ourexperimental results across two datasets for two IR tasks reveal that retrievalpipelines are not robust to these query variations, with effectiveness drops of$\approx20\%$ on average. The code and datasets are available athttps://github.com/Guzpenha/query_variation_generators.

Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators