Benchmarks

This is a short effort to give users an idea of how long the functions take to process. The benchmarks were performed using the default R install on Travis CI.

Note: the benchmarks run significantly faster on my personal machine (MacBook Pro Late 2016). In most cases, the processes take \(\approx\) 25% of the time.

We will be estimating a tri-diagonal precision matrix with dimension \(p = 100\):

library(SCPME)
library(microbenchmark)

#  generate data from tri-diagonal (sparse) matrix
data = data_gen(p = 100, n = 1000, r = 5)

# calculate sample covariance matrix
sample = (nrow(data$X) - 1)/nrow(data$X)*cov(data$X)

Default convergence tolerance with specified tuning parameters (no cross validation):

# benchmark shrink - default tolerance
microbenchmark(shrink(S = sample, crit.cv = "loglik", lam = 0.1, tol.abs = 1e-4, tol.rel = 1e-4, trace = "none"))

## Unit: milliseconds
##                                                                                                      expr
##  shrink(S = sample, crit.cv = "loglik", lam = 0.1, tol.abs = 1e-04,      tol.rel = 1e-04, trace = "none")
##       min       lq     mean   median       uq      max neval
##  443.3506 450.3688 463.6747 461.9791 473.2268 496.8634   100

Stricter convergence tolerance with specified tuning parameters (no cross validation):

# benchmark shrink - tolerance 1e-8
microbenchmark(shrink(S = sample, crit.cv = "loglik", lam = 0.1, tol.abs = 1e-8, tol.rel = 1e-8, trace = "none"))

## Unit: seconds
##                                                                                                      expr
##  shrink(S = sample, crit.cv = "loglik", lam = 0.1, tol.abs = 1e-08,      tol.rel = 1e-08, trace = "none")
##       min       lq     mean   median       uq      max neval
##  1.600358 1.646749 1.670888 1.669327 1.690355 1.778268   100

Default convergence tolerance with cross validation for lam:

# benchmark shrink CV - default parameter grid
microbenchmark(shrink(X = data$X, Y = data$Y, trace = "none"), times = 5)

## Unit: seconds
##                                            expr      min       lq     mean
##  shrink(X = data$X, Y = data$Y, trace = "none") 16.06304 16.36117 16.61059
##    median       uq      max neval
##  16.43248 16.60028 17.59596     5

Parallel (cores = 2) cross validation:

# benchmark shrink parallel CV
microbenchmark(shrink(X = data$X, Y = data$Y, cores = 2, trace = "none"), times = 5)

## Unit: seconds
##                                                       expr      min
##  shrink(X = data$X, Y = data$Y, cores = 2, trace = "none") 12.25516
##        lq     mean  median       uq      max neval
##  12.37918 12.50935 12.4228 12.65463 12.83499     5

Cross validation with \(B = \hat{\Sigma}_{xy}\):

# benchmark shrink penalizing beta
lam_max = max(abs(crossprod(data$X, data$Y)))
microbenchmark(shrink(X = data$X, Y = data$Y, B = cov(data$X, data$Y), lam.max = lam_max, lam.min.ratio = 1e-4, trace = "none"), times = 5)

## Unit: seconds
##                                                                                                                    expr
##  shrink(X = data$X, Y = data$Y, B = cov(data$X, data$Y), lam.max = lam_max,      lam.min.ratio = 1e-04, trace = "none")
##      min       lq     mean   median       uq      max neval
##  19.4061 19.87525 20.09298 20.10041 20.47427 20.60889     5

Cross validation with \(B = \left[\hat{\Sigma}_{xy}, I_{p} \right]\):

# benchmark shrink penalizing beta and omega
microbenchmark(shrink(X = data$X, Y = data$Y, B = cbind(cov(data$X, data$Y), diag(ncol(data$X))), lam.max = 10, lam.min.ratio = 1e-4, trace = "none"), times = 5)

## Unit: seconds
##                                                                                                                                               expr
##  shrink(X = data$X, Y = data$Y, B = cbind(cov(data$X, data$Y),      diag(ncol(data$X))), lam.max = 10, lam.min.ratio = 1e-04,      trace = "none")
##       min       lq     mean   median       uq     max neval
##  64.52161 65.65382 66.49351 65.99164 67.67487 68.6256     5

Matt Galloway