English (United Kingdom)

https://curated-unify.zendy.io/wp-json/zendy-region/v1/featured_content/oa?rat=en

https://curated-unify.zendy.io/wp-json/zendy-region/v1/highlighted_journal/

Global: Zendy Plus annual

Presents the access of premium content as premium feature

Premium Content

Presents the keyphrase highlighting as premium feature

Keyphrase Highlighting

Presents the summarisation as premium feature

Summarisation

Insights

Presents the pdf analysis as premium feature

PDF Analysis

Presents the zaia usage as premium feature

ZAIA

Global: Zendy Plus monthly

Global: Zendy Tools annual

Global: Zendy Tools monthly

Global: Zendy Free Plan

High-dimensional vector similarity search (HVSS) is gaining prominence as apowerful tool for various data science and AI applications. As vector datascales up, in-memory indexes pose a significant challenge due to thesubstantial increase in main memory requirements. A potential solution involvesleveraging disk-based implementation, which stores and searches vector data onhigh-performance devices like NVMe SSDs. However, implementing HVSS for datasegments proves to be intricate in vector databases where a single machinecomprises multiple segments for system scalability. In this context, eachsegment operates with limited memory and disk space, necessitating a delicatebalance between accuracy, efficiency, and space cost. Existing disk-basedmethods fall short as they do not holistically address all these requirementssimultaneously. In this paper, we present Starling, an I/O-efficientdisk-resident graph index framework that optimizes data layout and searchstrategy within the segment. It has two primary components: (1) a data layoutincorporating an in-memory navigation graph and a reordered disk-based graphwith enhanced locality, reducing the search path length and minimizing diskbandwidth wastage; and (2) a block search strategy designed to minimize costlydisk I/O operations during vector query execution. Through extensiveexperiments, we validate the effectiveness, efficiency, and scalability ofStarling. On a data segment with 2GB memory and 10GB disk capacity, Starlingcan accommodate up to 33 million vectors in 128 dimensions, offering HVSS withover 0.9 average precision and top-10 recall rate, and latency under 1millisecond. The results showcase Starling's superior performance, exhibiting43.9$\times$ higher throughput with 98% lower query latency compared tostate-of-the-art methods while maintaining the same level of accuracy.

Starling: An I/O-Efficient Disk-Resident Graph Index Framework for High-Dimensional Vector Similarity Search on Data Segment