Principal Component Analysis Based Filtering for Scalable, High Precision k-NN Search

Approximate <tex-math notation="LaTeX">k</tex-math> Nearest Neighbours (A <tex-math notation="LaTeX">k</tex-math> NN) search is widely used in domains such as computer vision and machine learning. However, A <tex-math notation="LaTeX">k </tex-math> NN search in high-dimensional datasets does not scale well on multicore platforms, due to its large memory footprint. Parallel A <tex-math notation="LaTeX">k</tex-math> NN search using space subdivision for filtering helps reduce the memory footprint, but its loss of precision is unstable. In this paper, we propose a new data filtering method—PCAF—for parallel A <tex-math notation="LaTeX">k</tex-math> NN search based on principal component analysis. PCAF improves on previous methods, demonstrating sustained, high scalability for a wide range of high-dimensional datasets on both Intel and AMD multicore platforms. Moreover, PCAF maintains highly precise A <tex-math notation="LaTeX">k</tex-math> NN search results.