BET on Independence

Kai Zhang* Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NCCONTACT Kai Zhang zhangk@email.unc.edu Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, Chapel Hill, NC 27599.* This article is dedicated to Professor Lawrence D. Brown with the author’s deepest appreciation for his teaching and mentoring.Color versions of one or more of the figures in the article can be found online at www.tandfonline.com/r/JASA.Supplementary materials for this article are available online. Please go to www.tandfonline.com/r/JASA.AbstractWe study the problem of nonparametric dependence detection. Many existing methods may suffer severe power loss due to nonuniform consistency, which we illustrate with a paradox. To avoid such power loss, we approach the nonparametric test of independence through the new framework of binary expansion statistics (BEStat) and binary expansion testing (BET), which examine dependence through a novel binary expansion filtration approximation of the copula. Through a Hadamard transform, we find that the symmetry statistics in the filtration are complete sufficient statistics for dependence. These statistics are also uncorrelated under the null. By using symmetry statistics, the BET avoids the problem of nonuniform consistency and improves upon a wide class of commonly used methods (a) by achieving the minimax rate in sample size requirement for reliable power and (b) by providing clear interpretations of global relationships upon rejection of independence. The binary expansion approach also connects the symmetry statistics with the current computing system to facilitate efficient bitwise implementation. We illustrate the BET with a study of the distribution of stars in the night sky and with an exploratory data analysis of the TCGA breast cancer data. Supplementary materials for this article are available online.