picasso: Sparse Learning with Convex and Non-Convex Penalties

Description

Computationally efficient tools for fitting sparse generalized linear models with convex or non-convex penalties. Supported penalties include lasso, smoothly clipped absolute deviation (SCAD), and minimax concave penalty (MCP). Computation is based on multi-stage convex relaxation and pathwise coordinate optimization with warm starts, active-set updates, and screening rules.

Details

Core optimization routines are implemented in C++ for speed, and coefficient paths are stored as sparse matrices for memory efficiency.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso.

Extract Coefficients from an Object with S3 Class "gaussian"

Description

Extract estimated coefficient vectors from selected points on the regularization path.

Usage

## S3 method for class 'gaussian'
coef(object, lambda.idx = c(1:3), beta.idx = c(1:3), ...)

Arguments

Value

A numeric matrix of extracted coefficients. Columns correspond to lambda.idx. Rows contain the intercept ("(Intercept)") and the selected coefficients ("beta[i]" for i in beta.idx).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Extract Coefficients from an Object with S3 Class "logit"

Description

Extract estimated coefficient vectors from selected points on the regularization path.

Usage

## S3 method for class 'logit'
coef(object, lambda.idx = c(1:3), beta.idx = c(1:3), ...)

Arguments

Value

A numeric matrix of extracted coefficients. Columns correspond to lambda.idx. Rows contain the intercept ("(Intercept)") and the selected coefficients ("beta[i]" for i in beta.idx).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Extract Coefficients from an Object with S3 Class "poisson"

Description

Extract estimated coefficient vectors from selected points on the regularization path.

Usage

## S3 method for class 'poisson'
coef(object, lambda.idx = c(1:3), beta.idx = c(1:3), ...)

Arguments

Value

A numeric matrix of extracted coefficients. Columns correspond to lambda.idx. Rows contain the intercept ("(Intercept)") and the selected coefficients ("beta[i]" for i in beta.idx).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Extract Coefficients from an Object with S3 Class "sqrtlasso"

Description

Extract estimated coefficient vectors from selected points on the regularization path.

Usage

## S3 method for class 'sqrtlasso'
coef(object, lambda.idx = c(1:3), beta.idx = c(1:3), ...)

Arguments

Value

A numeric matrix of extracted coefficients. Columns correspond to lambda.idx. Rows contain the intercept ("(Intercept)") and the selected coefficients ("beta[i]" for i in beta.idx).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Bardet-Biedl Syndrome Gene Expression Data from Scheetz et al. (2006)

Description

Gene expression data (200 genes, 120 samples) from microarray experiments on mammalian eye tissue reported by Scheetz et al. (2006).

Usage

data(eyedata)

Format

A list with components:

x

A 120 by 200 matrix of gene probes.

y

A numeric response vector of length 120 representing the expression level of the TRIM32 gene.

Details

This data set contains 120 samples and 200 predictors.

Author(s)

Xingguo Li, Tuo Zhao, Tong Zhang and Han Liu
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

References

1. T. Scheetz, K. Kim, R. Swiderski, A. Philp, T. Braun, K. Knudtson, A. Dorrance, G. DiBona, J. Huang, T. Casavant, V. Sheffield, and E. Stone. Regulation of gene expression in the mammalian eye and its relevance to eye disease. Proceedings of the National Academy of Sciences of the United States of America, 2006.

See Also

picasso-package.

Examples

data(eyedata)
image(eyedata$x)

Fit Sparse Regression Models with Convex and Non-Convex Penalties

Description

picasso fits regularization paths for sparse Gaussian, logistic (family = "binomial"), Poisson, and sqrt-lasso models using lasso, MCP, or SCAD penalties.

Usage

picasso(X, Y, lambda = NULL, nlambda = 100, lambda.min.ratio =
                 0.05, family = "gaussian", method = "l1",
                 type.gaussian = "naive", gamma = 3, df = NULL,
                 standardize = TRUE, intercept = TRUE, prec = 1e-07,
                 max.ite = 1000, verbose = FALSE)

Arguments

Details

When lambda is not supplied, picasso constructs a logarithmically spaced path between \lambda_{\max} and \lambda_{\min} = \mathrm{lambda.min.ratio} \cdot \lambda_{\max}.

The method solves a penalized optimization problem:

\min_{\beta_0, \beta}\; \mathcal{L}(\beta_0,\beta) + \sum_{j=1}^d p_{\lambda,\gamma}(|\beta_j|),

where p_{\lambda,\gamma} is lasso, MCP, or SCAD.

Loss functions by family are:

• "gaussian": \mathcal{L}(\beta_0,\beta)=\frac{1}{2n}\lVert Y-\beta_0\mathbf{1}-X\beta \rVert_2^2.

• "binomial": \mathcal{L}(\beta_0,\beta)=\frac{1}{n}\sum_{i=1}^n[\log(1+\exp(\beta_0+x_i^\top\beta))-y_i(\beta_0+x_i^\top\beta)].

• "poisson": \mathcal{L}(\beta_0,\beta)=\frac{1}{n}\sum_{i=1}^n[\exp(\beta_0+x_i^\top\beta)-y_i(\beta_0+x_i^\top\beta)].

• "sqrtlasso": \mathcal{L}(\beta_0,\beta)=\frac{1}{\sqrt{n}}\lVert Y-\beta_0\mathbf{1}-X\beta \rVert_2.

Value

A fitted object of class "gaussian", "logit", "poisson", or "sqrtlasso" depending on family.

Common components include:

beta

Sparse matrix of fitted coefficients. Columns correspond to regularization parameters in the solution path.

intercept

Intercept values along the solution path.

lambda

Regularization sequence used for fitting.

nlambda

Number of regularization values.

df

Degrees of freedom (number of nonzero coefficients) along the path.

method

Penalty type used for fitting.

ite

Iteration information returned by the solver.

verbose

Input verbose value.

runtime

Elapsed fitting time.

family

Input family value.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

References

1. J. Friedman, T. Hastie, H. Hofling, and R. Tibshirani. Pathwise coordinate optimization. The Annals of Applied Statistics, 2007.
2. C.-H. Zhang. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 2010.
3. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 2001.
4. R. Tibshirani, J. Bien, J. Friedman, T. Hastie, N. Simon, J. Taylor, and R. Tibshirani. Strong rules for discarding predictors in lasso-type problems. Journal of the Royal Statistical Society: Series B, 2012.
5. J. Ge, X. Li, H. Jiang, M. Wang, T. Zhang, H. Liu, and T. Zhao. PICASSO: A sparse learning library for high-dimensional data analysis in R and Python. arXiv preprint https://arxiv.org/abs/2006.15261.

See Also

picasso-package.

Examples

################################################################
## Sparse linear regression
## Generate the design matrix and regression coefficient vector
n = 100 # sample number 
d = 80 # sample dimension
c = 0.5 # correlation parameter
s = 20  # support size of coefficient
set.seed(2016)
X = scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n)
beta = c(runif(s), rep(0, d-s))

## Generate response using Gaussian noise, and fit sparse linear models
noise = rnorm(n)
Y = X%*%beta + noise

## l1 regularization solved with naive update
fitted.l1.naive = picasso(X, Y, nlambda=100, type.gaussian="naive")

## backward-compatible covariance alias (same backend as "naive")
fitted.l1.covariance  = picasso(X, Y, nlambda=100, type.gaussian="covariance")

## mcp regularization
fitted.mcp = picasso(X, Y, nlambda=100, method="mcp")

## scad regularization
fitted.scad = picasso(X, Y, nlambda=100, method="scad")

## lambdas used 
print(fitted.l1.naive$lambda)

## number of nonzero coefficients for each lambda
print(fitted.l1.naive$df)

## coefficients and intercept for the i-th lambda
i = 30
print(fitted.l1.naive$lambda[i])
print(fitted.l1.naive$beta[,i])
print(fitted.l1.naive$intercept[i])

## Visualize the solution path
plot(fitted.l1.naive)
plot(fitted.l1.covariance)
plot(fitted.mcp)
plot(fitted.scad)


################################################################
## Sparse logistic regression
## Generate the design matrix and regression coefficient vector
n <- 100  # sample number 
d <- 80   # sample dimension
c <- 0.5   # parameter controlling the correlation between columns of X
s <- 20    # support size of coefficient
set.seed(2016)
X <- scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n)
beta <- c(runif(s), rep(0, d-s))

## Generate response and fit sparse logistic models
p = 1/(1+exp(-X%*%beta))
Y = rbinom(n, rep(1,n), p)

## l1 regularization
fitted.l1 = picasso(X, Y, nlambda=100, family="binomial", method="l1")

## mcp regularization
fitted.mcp = picasso(X, Y, nlambda=100, family="binomial", method="mcp")

## scad regularization
fitted.scad = picasso(X, Y, nlambda=100, family="binomial", method="scad")

## lambdas used 
print(fitted.l1$lambda)

## number of nonzero coefficients for each lambda
print(fitted.l1$df)

## coefficients and intercept for the i-th lambda
i = 30
print(fitted.l1$lambda[i])
print(fitted.l1$beta[,i])
print(fitted.l1$intercept[i])

## Visualize the solution path
plot(fitted.l1)

## Estimate of Bernoulli parameters
param.l1 = fitted.l1$p


################################################################
## Sparse poisson regression
## Generate the design matrix and regression coefficient vector
n <- 100  # sample number 
d <- 80   # sample dimension
c <- 0.5   # parameter controlling the correlation between columns of X
s <- 20    # support size of coefficient
set.seed(2016)
X <- scale(matrix(rnorm(n*d),n,d)+c*rnorm(n))/sqrt(n-1)*sqrt(n)
beta <- c(runif(s), rep(0, d-s))/sqrt(s)

## Generate response and fit sparse poisson models
p = X%*%beta+rnorm(n)
Y = rpois(n, exp(p))

## l1 regularization
fitted.l1 = picasso(X, Y, nlambda=100, family="poisson", method="l1")

## mcp regularization
fitted.mcp = picasso(X, Y, nlambda=100, family="poisson", method="mcp")

## scad regularization
fitted.scad = picasso(X, Y, nlambda=100, family="poisson", method="scad")

## lambdas used 
print(fitted.l1$lambda)

## number of nonzero coefficients for each lambda
print(fitted.l1$df)

## coefficients and intercept for the i-th lambda
i = 30
print(fitted.l1$lambda[i])
print(fitted.l1$beta[,i])
print(fitted.l1$intercept[i])

## Visualize the solution path
plot(fitted.l1)

Plot Method for an Object with S3 Class "gaussian"

Description

Plot coefficient paths across regularization parameters.

Usage

## S3 method for class 'gaussian'
plot(x, ...)

Arguments

Value

No return value, called for side effects (drawing the coefficient-path plot).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Plot Method for an Object with S3 Class "logit"

Description

Plot coefficient paths across regularization parameters.

Usage

## S3 method for class 'logit'
plot(x, ...)

Arguments

Value

No return value, called for side effects (drawing the coefficient-path plot).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Plot Method for an Object with S3 Class "poisson"

Description

Plot coefficient paths across regularization parameters.

Usage

## S3 method for class 'poisson'
plot(x, ...)

Arguments

Value

No return value, called for side effects (drawing the coefficient-path plot).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Plot Method for an Object with S3 Class "sqrtlasso"

Description

Plot coefficient paths across regularization parameters.

Usage

## S3 method for class 'sqrtlasso'
plot(x, ...)

Arguments

Value

No return value, called for side effects (drawing the coefficient-path plot).

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Prediction Method for an Object with S3 Class "gaussian"

Description

Predict responses for new data using fitted models.

Usage

## S3 method for class 'gaussian'
predict(object, newdata, lambda.idx = c(1:3), Y.pred.idx = c(1:5), ...)

Arguments

Details

predict.gaussian returns predicted responses for newdata using fitted coefficients from object:

\hat{Y} = \hat{\beta}_0 + X_{new} \hat{\beta}.

Value

Numeric matrix of predicted responses. Rows correspond to observations in newdata; columns correspond to lambda.idx.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Prediction Method for an Object with S3 Class "logit"

Description

Predict responses for new data using fitted models.

Usage

## S3 method for class 'logit'
predict(object, newdata, lambda.idx = c(1:3), p.pred.idx = c(1:5), ...)

Arguments

Details

predict.logit returns predicted Bernoulli probabilities for newdata using fitted coefficients from object:

\hat{p} = \frac{e^{\hat{\beta}_0 + X_{new} \hat{\beta}}}{1+e^{\hat{\beta}_0 + X_{new} \hat{\beta}}}.

Value

Numeric matrix of predicted Bernoulli probabilities. Rows correspond to observations in newdata; columns correspond to lambda.idx.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Prediction Method for an Object with S3 Class "poisson"

Description

Predict responses for new data using fitted models.

Usage

## S3 method for class 'poisson'
predict(object, newdata, lambda.idx = c(1:3), p.pred.idx = c(1:5), ...)

Arguments

Details

predict.poisson returns predicted Poisson means for newdata using fitted coefficients from object:

\hat{p} = e^{\hat{\beta}_0 + X_{new} \hat{\beta}}.

Value

Numeric matrix of predicted Poisson means. Rows correspond to observations in newdata; columns correspond to lambda.idx.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Prediction Method for an Object with S3 Class "sqrtlasso"

Description

Predict responses for new data using fitted models.

Usage

## S3 method for class 'sqrtlasso'
predict(object, newdata, lambda.idx = c(1:3), Y.pred.idx = c(1:5), ...)

Arguments

Details

predict.sqrtlasso returns predicted responses for newdata using fitted coefficients from object:

\hat{Y} = \hat{\beta}_0 + X_{new} \hat{\beta}.

Value

Numeric matrix of predicted responses. Rows correspond to observations in newdata; columns correspond to lambda.idx.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Print Method for an Object with S3 Class "gaussian"

Description

Print a concise summary for an object with S3 class "gaussian".

Usage

## S3 method for class 'gaussian'
print(x, ...)

Arguments

Details

This method prints tuning parameters and fit statistics for a fitted model object.

Value

The input object x, returned invisibly.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Print Method for an Object with S3 Class "logit"

Description

Print a concise summary for an object with S3 class "logit".

Usage

## S3 method for class 'logit'
print(x, ...)

Arguments

Details

This method prints tuning parameters and fit statistics for a fitted logistic model object.

Value

The input object x, returned invisibly.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Print Method for an Object with S3 Class "poisson"

Description

Print a concise summary for an object with S3 class "poisson".

Usage

## S3 method for class 'poisson'
print(x, ...)

Arguments

Details

This method prints tuning parameters and fit statistics for a fitted Poisson model object.

Value

The input object x, returned invisibly.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.

Print Method for an Object with S3 Class "sqrtlasso"

Description

Print a concise summary for an object with S3 class "sqrtlasso".

Usage

## S3 method for class 'sqrtlasso'
print(x, ...)

Arguments

Details

This method prints tuning parameters and fit statistics for a fitted model object.

Value

The input object x, returned invisibly.

Author(s)

Jason Ge, Xingguo Li, Haoming Jiang, Mengdi Wang, Tong Zhang, Han Liu and Tuo Zhao
Maintainer: Tuo Zhao <tourzhao@gatech.edu>

See Also

picasso and picasso-package.