HNSW 색인을 위한 LSH 기반 대량 적재 기법

초록

Hierarchical Navigable Small World (HNSW) is an indexing structure used for nearest neighbor vector searches. It is being actively used in recent vector databases. However, the construction of an HNSW index from a large number of vectors is very time-consuming due to repeated addition of individual vectors to the HNSW index. To address this issue, this paper proposed a bulk-loading method that could accelerate the construction of HNSW indices from large-scale vectors by utilizing the locality-sensitive hashing (LSH) technique. The proposed method could quickly identify neighboring vectors using LSH and directly connect them to efficiently construct the HNSW index. Experimental results demonstrated that the proposed method significantly reduced the index construction time compared to the conventional individual vector insertion approach while improving search accuracy and maintaining a similar level of search speed.

키워드