Logistic Regression

Computes the coefficient estimates for logistic regression. ridge regularization and bridge regularization optional.

logisticr(X, y, lam = seq(0, 2, 0.1), alpha = 1.5, penalty = c("none",
  "ridge", "bridge"), intercept = TRUE, method = c("IRLS", "MM"),
  tol = 1e-05, maxit = 1e+05, vec = NULL, init = 1,
  criteria = c("logloss", "mse", "misclass"), K = 5)

Arguments

X	matrix or data frame
y	matrix or vector of response values 0,1
lam	optional tuning parameter(s) for ridge regularization term. If passing a list of values, the function will choose optimal value based on K-fold cross validation. Defaults to `lam = seq(0, 2, 0.1)`
alpha	optional tuning parameter for bridge regularization term. If passing a list of values, the function will choose the optimal value based on K-fold cross validation. Defaults to 'alpha = 1.5'
penalty	choose from c('none', 'ridge', 'bridge'). Defaults to 'none'
intercept	Defaults to TRUE
method	optimization algorithm. Choose from 'IRLS' or 'MM'. Defaults to 'IRLS'
tol	tolerance - used to determine algorithm convergence. Defaults to 10^-5
maxit	maximum iterations. Defaults to 10^5
vec	optional vector to specify which coefficients will be penalized
init	optional initialization for MM algorithm
criteria	specify the criteria for cross validation. Choose from c('mse', 'logloss', 'misclass'). Defauls to 'logloss'
K	specify number of folds for cross validation, if necessary

Value

returns selected tuning parameters, beta estimates (includes intercept), MSE, log loss, misclassification rate, total iterations, and gradients.

Examples

library(dplyr)
X = dplyr::select(iris, -Species)
y = dplyr::select(iris, Species)
y$Species = ifelse(y$Species == 'setosa', 1, 0)
logisticr(X, y)
#> 
#> Call: logisticr(X = X, y = y)
#> 
#> Iterations:
#> [1] 18
#> 
#> Tuning parameters:
#>       lam  alpha
#> [1,]  NaN    NaN
#> 
#> MSE:
#> [1] 7.707778e-14
#> 
#> logloss:
#> [1] 6.64746e-06
#> 
#> misclassification:
#> [1] 0
#> 
#> Coefficients:
#>                [,1]
#> intercept -12.11317
#>             6.99937
#>             6.01011
#>           -12.38368
#>           -13.25003

# ridge Logistic Regression with IRLS
logisticr(X, y, lam = 0.1, penalty = 'ridge')
#> 
#> Call: logisticr(X = X, y = y, lam = 0.1, penalty = "ridge")
#> 
#> Iterations:
#> [1] 11
#> 
#> Tuning parameters:
#>       lam  alpha
#> [1,]  0.1    NaN
#> 
#> MSE:
#> [1] 5.499678e-05
#> 
#> logloss:
#> [1] 0.3940251
#> 
#> misclassification:
#> [1] 0
#> 
#> Coefficients:
#>               [,1]
#> intercept  8.34036
#>           -0.49766
#>            1.61806
#>           -3.54530
#>           -1.60032

# ridge Logistic Regression with MM
logisticr(X, y, lam = 0.1, penalty = 'ridge', method = 'MM')
#> 
#> Call: logisticr(X = X, y = y, lam = 0.1, penalty = "ridge", method = "MM")
#> 
#> Iterations:
#> [1] 7028
#> 
#> Tuning parameters:
#>       lam  alpha
#> [1,]  0.1    NaN
#> 
#> MSE:
#> [1] 5.499753e-05
#> 
#> logloss:
#> [1] 0.3940282
#> 
#> misclassification:
#> [1] 0
#> 
#> Coefficients:
#>               [,1]
#> intercept  8.34027
#>           -0.49765
#>            1.61807
#>           -3.54529
#>           -1.60032