English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

Sparse data are common. The traditional ``handcrafted'' features are oftensparse. Embedding vectors from trained models can also be very sparse, forexample, embeddings trained via the ``ReLu'' activation function. In thispaper, we report our exploration of efficient search in sparse data withgraph-based ANN algorithms (e.g., HNSW, or SONG which is the GPU version ofHNSW), which are popular in industrial practice, e.g., search and ads(advertising).  We experiment with the proprietary ads targeting application, as well asbenchmark public datasets. For ads targeting, we train embeddings with thestandard ``cosine two-tower'' model and we also develop the ``chi-squaretwo-tower'' model. Both models produce (highly) sparse embeddings when they areintegrated with the ``ReLu'' activation function. In EBR (embedding-basedretrieval) applications, after we the embeddings are trained, the next crucialtask is the approximate near neighbor (ANN) search for serving. While there aremany ANN algorithms we can choose from, in this study, we focus on thegraph-based ANN algorithm (e.g., HNSW-type).  Sparse embeddings should help improve the efficiency of EBR. One benefit isthe reduced memory cost for the embeddings. The other obvious benefit is thereduced computational time for evaluating similarities, because, forgraph-based ANN algorithms such as HNSW, computing similarities is often thedominating cost. In addition to the effort on leveraging data sparsity forstorage and computation, we also integrate ``sign cauchy random projections''(SignCRP) to hash vectors to bits, to further reduce the memory cost and speedup the ANN search. In NIPS'13, SignCRP was proposed to hash the chi-squaresimilarity, which is a well-adopted nonlinear kernel in NLP and computervision. Therefore, the chi-square two-tower model, SignCRP, and HNSW are nowtightly integrated.

Practice with Graph-based ANN Algorithms on Sparse Data: Chi-square Two-tower model, HNSW, Sign Cauchy Projections