A major goal of speciation genomic studies is to infer the basis of reproductive isolation from patterns of genome-wide variation. However, Interpreting the genomic landscape of speciation is difficult because patterns of variation can be shaped by many factors, some of which have nothing to do with speciation, per se. Recent work has suggested that background selection (BGS) is an important confounding factor that may lead to erroneous conclusions about the genomic location of barrier loci. In response, several authors have proposed methods for controlling for its effect. One such method involves using the correlation in differentiation across multiple closely related taxa as a baseline in outlier detectionthe assumption that correlated genomic landscapes arise primarily due to BGS. Using empirical and simulated data from a radiation of monkeyflowers, I will argue that (i) BGS is unlikely to be major factor confounding speciation genomic studies and (ii) in attempting to control for it, we may actually be discarding the polygenic signature of adaptation and speciation.