The Strongest Correlations Among Many Variables, With a GPU

Finds the strongly correlated pairs among the columns of x (or between the columns of x and those of y) without forming the whole correlation matrix: every pair whose correlation reaches a threshold, or each variable's strongest partners. It is for data too wide for gpuCor or cor: tens of thousands of genes, whose correlation matrix would take gigabytes, reduced to the edge list of a co-expression network.

Usage

gpuCorPairs(x, y = NULL, threshold = NULL, top = NULL,
    method = "pearson", absolute = TRUE, test = FALSE, limit = 1e7,
    tile = 2048L, precision = getOption("gputools.precision", "single"))

Arguments

x

a numeric matrix whose columns are the variables, with no missing values.

y

a numeric matrix with the same number of rows, or NULL for x with itself.

threshold

keep every pair whose correlation is at least this strong.

top

or instead, keep each column of y's top strongest partners among the columns of x. Give one of threshold and top.

method

"pearson" or "spearman".

absolute

TRUE to measure strength by \(|r|\), so that strong negative correlations count; FALSE to keep positive ones only.

test

TRUE to add t statistics and two-sided p-values, as gpuCor computes them.

limit

the most pairs to return: a threshold that lets more through is an error, not a result of unexpected size. Raise it if you want them all.

tile

the GPU computes the correlation matrix in blocks of tile x tile. The default suits GPUs with a few gigabytes free; it has no effect on the result.

precision

"single" or "double": the floating point precision the GPU computes in; see gpuCor.

Details

The correlations are the ones cor computes (Spearman's is Pearson's on the ranks, ties averaged). A constant column correlates with nothing, as its correlations are NA, and is never returned.

With y = NULL, a threshold returns each pair once, with i < j; top returns each column's partners, so a pair that is in both columns' lists appears twice, and a column is never its own partner.

A p-value cut-off \(\alpha\) is a threshold on \(|r|\), since every pair has the same \(n\): \(|r| \ge t / \sqrt{n - 2 + t^2}\) with \(t\) the \(1 - \alpha/2\) quantile of the t distribution on \(n - 2\) degrees of freedom. Remember how many pairs are tested when choosing \(\alpha\).

The whole of x (and y) is copied to the GPU, in the precision computed in; the correlation matrix itself never is.

Value

A data frame, a row per pair:

i

the column of x.

j

the column of y (of x when y is NULL).

r

their correlation.

x, y

the columns' names, if they have them.

t, p.value

with test = TRUE.

Rows are ordered by j, then by i for a threshold, and from strongest to weakest for top.

See also

gpuCor for the whole matrix.

Examples

if (FALSE) { # gpuAvailable()
set.seed(1)
genes <- matrix(rnorm(50 * 2000), 50, 2000,
                dimnames = list(NULL, paste0("g", 1:2000)))
genes[, 2] <- genes[, 1] + rnorm(50, sd = 0.1)

# every pair correlated at |r| >= 0.6
head(gpuCorPairs(genes, threshold = 0.6))

# each gene's three strongest partners, with p-values
head(gpuCorPairs(genes, top = 3, test = TRUE))
}