English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

While state-of-the-art NLP explainability (XAI) methods focus on explainingper-sample decisions in supervised end or probing tasks, this is insufficientto explain and quantify model knowledge transfer during (un-)supervisedtraining. Thus, for TX-Ray, we modify the established computer visionexplainability principle of 'visualizing preferred inputs of neurons' to makeit usable transfer analysis and NLP. This allows one to analyze, track andquantify how self- or supervised NLP models first build knowledge abstractionsin pretraining (1), and then transfer these abstractions to a new domain (2),or adapt them during supervised fine-tuning (3). TX-Ray expresses neurons asfeature preference distributions to quantify fine-grained knowledge transfer oradaptation and guide human analysis. We find that, similar to Lottery Ticketbased pruning, TX-Ray based pruning can improve test set generalization andthat it can reveal how early stages of self-supervision automatically learnlinguistic abstractions like parts-of-speech.

TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP