English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

The Word Mover's Distance (WMD) is a metric that measures the semanticdissimilarity between two text documents by computing the cost of moving allwords of a source/query document to the most similar words of a target documentoptimally. Computing WMD between two documents is costly because it requiressolving an optimization problem that costs \(O(V^3log(V))\) where \(V\) is thenumber of unique words in the document. Fortunately, the WMD can be framed asthe Earth Mover's Distance (EMD) (also known as the Optimal TransportationDistance) for which it has been shown that the algorithmic complexity can bereduced to \(O(V^2)\) by adding an entropy penalty to the optimization problemand a similar idea can be adapted to compute WMD efficiently. Additionally, thecomputation can be made highly parallel by computing WMD of a single querydocument against multiple target documents at once (e.g., finding whether agiven tweet is similar to any other tweets happened in a day). In this paper,we present a shared-memory parallel Sinkhorn-Knopp Algorithm to compute the WMDof one document against many other documents by adopting the \(O(V^2)\) EMDalgorithm. We used algorithmic transformations to change the original densecompute-heavy kernel to a sparse compute kernel and obtained \(67\times\)speedup using \(96\) cores on the state-of-the-art of Intel\textregistered{}4-sockets Cascade Lake machine w.r.t. its sequential run. Our parallelalgorithm is over \(700\times\) faster than the naive parallel python code thatinternally uses optimized matrix library calls.

An Efficient Shared-memory Parallel Sinkhorn-Knopp Algorithm to Compute the Word Mover's Distance