Sampling a Near Neighbor in High Dimensions-Who is the Fairest of Them All?
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Sampling a Near Neighbor in High Dimensions-Who is the Fairest of Them All? / Aumüller, Martin; Har-Peled, Sariel; Mahabadi, Sepideh; Pagh, Rasmus; Silvestri, Francesco.
In: ACM Transactions on Database Systems, Vol. 47, No. 1, 4, 2022, p. 1-40.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Sampling a Near Neighbor in High Dimensions-Who is the Fairest of Them All?
AU - Aumüller, Martin
AU - Har-Peled, Sariel
AU - Mahabadi, Sepideh
AU - Pagh, Rasmus
AU - Silvestri, Francesco
N1 - Publisher Copyright: © 2022 Association for Computing Machinery.
PY - 2022
Y1 - 2022
N2 - Similarity search is a fundamental algorithmic primitive, widely used in many computer science disciplines. Given a set of points S and a radius parameter r > 0, the r-near neighbor (r-NN) problem asks for a data structure that, given any query point q, returns a point p within distance at most r from q. In this paper, we study the r-NN problem in the light of individual fairness and providing equal opportunities: all points that are within distance r from the query should have the same probability to be returned. In the low-dimensional case, this problem was first studied by Hu, Qiao, and Tao (PODS 2014). Locality sensitive hashing (LSH), the theoretically strongest approach to similarity search in high dimensions, does not provide such a fairness guarantee.In this work, we show that LSH based algorithms can be made fair, without a significant loss in efficiency. We propose several efficient data structures for the exact and approximate variants of the fair NN problem. Our approach works more generally for sampling uniformly from a sub-collection of sets of a given collection and can be used in a few other applications. We also develop a data structure for fair similarity search under inner product that requires nearly-linear space and exploits locality sensitive filters. The paper concludes with an experimental evaluation that highlights the unfairness of state-of-the-art NN data structures and shows the performance of our algorithms on real-world datasets.
AB - Similarity search is a fundamental algorithmic primitive, widely used in many computer science disciplines. Given a set of points S and a radius parameter r > 0, the r-near neighbor (r-NN) problem asks for a data structure that, given any query point q, returns a point p within distance at most r from q. In this paper, we study the r-NN problem in the light of individual fairness and providing equal opportunities: all points that are within distance r from the query should have the same probability to be returned. In the low-dimensional case, this problem was first studied by Hu, Qiao, and Tao (PODS 2014). Locality sensitive hashing (LSH), the theoretically strongest approach to similarity search in high dimensions, does not provide such a fairness guarantee.In this work, we show that LSH based algorithms can be made fair, without a significant loss in efficiency. We propose several efficient data structures for the exact and approximate variants of the fair NN problem. Our approach works more generally for sampling uniformly from a sub-collection of sets of a given collection and can be used in a few other applications. We also develop a data structure for fair similarity search under inner product that requires nearly-linear space and exploits locality sensitive filters. The paper concludes with an experimental evaluation that highlights the unfairness of state-of-the-art NN data structures and shows the performance of our algorithms on real-world datasets.
KW - fairness
KW - locality sensitive hashing
KW - near neighbor
KW - sampling
KW - Similarity search
UR - http://www.scopus.com/inward/record.url?scp=85129852069&partnerID=8YFLogxK
U2 - 10.1145/3502867
DO - 10.1145/3502867
M3 - Journal article
AN - SCOPUS:85129852069
VL - 47
SP - 1
EP - 40
JO - ACM Transactions on Database Systems
JF - ACM Transactions on Database Systems
SN - 0362-5915
IS - 1
M1 - 4
ER -
ID: 340698972