English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Recently CKY-based models show great potential in unsupervised grammarinduction thanks to their human-like encoding paradigm, which runs recursivelyand hierarchically, but requires $O(n^3)$ time-complexity. RecursiveTransformer based on Differentiable Trees (R2D2) makes it possible to scale tolarge language model pre-training even with complex tree encoder by introducinga heuristic pruning method. However, the rule-based pruning approach suffersfrom local optimum and slow inference issues. In this paper, we fix thoseissues in a unified method. We propose to use a top-down parser as amodel-based pruning method, which also enables parallel encoding duringinference. Typically, our parser casts parsing as a split point scoring task,which first scores all split points for a given sentence, and then recursivelysplits a span into two by picking a split point with the highest score in thecurrent span. The reverse order of the splits is considered as the order ofpruning in R2D2 encoder. Beside the bi-directional language model loss, we alsooptimize the parser by minimizing the KL distance between tree probabilitiesfrom parser and R2D2. Our experiments show that our Fast-R2D2 improvesperformance significantly in grammar induction and achieves competitive resultsin downstream classification tasks.

Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation