Stepwise Gaussian graphical model selection

Written by Mohammed Saqr and Sonsoles López-Pernas

A partial correlation network describes which variables remain associated after all other variables are held constant. The regularized estimator most often used in psychology, the graphical lasso, selects the network and estimates its edges in one step, and its penalty shrinks every retained edge toward zero. When the aim is to interpret the size of the partial correlations, this shrinkage is a bias. Stepwise model selection separates the two tasks: it first chooses which edges belong in the network by comparing candidate graphs with an information criterion, and then estimates the partial correlations of the chosen graph by unpenalized maximum likelihood (Williams et al., 2019). The edges are therefore not shrunk, and sparsity comes from the choice of the graph rather than from a penalty.

This article selects a network for five self-regulated learning constructs by stepwise search, reads the unshrunk edges, compares them with the graphical lasso on the same data, describes the nodes through centrality and predictability, and checks the selection against rank-based correlations.

library(psychnets)

Data

SRL_GPT holds 300 sets of responses to the Motivated Strategies for Learning Questionnaire, each a construct score on a scale from 1 to 7. The responses were generated by a large language model (GPT) in a study of how such models answer psychological surveys (Saqr, 2025); they serve as a well-structured example and are not evidence about students.

head(SRL_GPT)
#>        CSU       IV       SE       SR   TA
#> 1 5.307692 5.666667 5.777778 5.333333 4.00
#> 2 5.846154 6.444444 6.000000 5.777778 4.00
#> 3 6.615385 6.666667 6.222222 6.333333 3.25
#> 4 5.692308 6.555556 6.333333 5.555556 4.50
#> 5 4.384615 5.555556 4.888889 4.777778 4.00
#> 6 4.846154 5.444444 5.666667 5.111111 3.50

With 300 observations and five variables, the sample is large relative to the number of parameters, which is the setting in which unregularized estimation is stable.

The model

The model is the Gaussian graphical model: under multivariate normality with precision matrix \(\mathbf{K} = \boldsymbol\Sigma^{-1}\), an edge between \(X_i\) and \(X_j\) is a non-zero partial correlation

\[\rho_{ij \cdot \mathrm{rest}} = -\frac{K_{ij}}{\sqrt{K_{ii}K_{jj}}}.\]

Candidate graphs are scored by the extended Bayesian information criterion,

\[\operatorname{EBIC}_{\gamma} = -2\ell(\widehat{\mathbf{K}}) + E\log n + 4E\gamma\log p,\]

where \(\ell\) is the log-likelihood of the graph refitted without a penalty, \(E\) its number of edges, \(n\) the sample size and \(p\) the number of nodes. The default \(\gamma = 0\) reduces the criterion to the ordinary BIC.

The search proceeds in two stages. A graphical-lasso path supplies candidate graphs ranging from empty to complete; each is refitted by maximum likelihood and scored, and the best becomes the starting graph. A stepwise search then adds or removes one edge at a time whenever the change lowers the criterion, and stops when no single change improves it. The result is a local optimum among graphs that differ by one edge.

Which constructs are directly associated?

psychnet() with method = "ggm" runs the stepwise selection.

stepwise_net <- psychnet(data = SRL_GPT, method = "ggm")
summary(stepwise_net)
#> <psychnet> ggm network
#>   nodes: 5   edges: 9   (undirected)
#>   optimality (KKT residual): 4.44e-16
#>   edge weight: range [-0.370, 0.422], mean 0.196

The selected graph has 9 of the 10 possible edges, with weights from -0.370 to 0.422. The edge table gives each unshrunk partial correlation:

as.data.frame(stepwise_net)
#>   from to     weight
#> 1  CSU IV  0.4215770
#> 2  CSU SE  0.3921754
#> 3   IV SE  0.1416604
#> 4  CSU SR  0.3385710
#> 5   IV SR  0.3333520
#> 6   SE SR  0.2173406
#> 7   IV TA  0.1565223
#> 8   SE TA  0.1368928
#> 9   SR TA -0.3699703

The learning constructs form a positive cluster. The strongest edge joins cognitive strategy use and intrinsic value (0.422), followed by cognitive strategy use with self-efficacy (0.392): respondents with higher strategy use report more intrinsic value and self-efficacy even when the other constructs are equal. Self-regulation and test anxiety share the only negative edge (-0.370). Test anxiety also has two modest positive edges, with intrinsic value (0.157) and self-efficacy (0.137). The CSU–TA pair is removed: the selected graph contains no association between strategy use and test anxiety beyond what the other constructs account for, although their ordinary correlation is not zero.

Is the refitted network the maximum-likelihood solution?

The final network is the maximum-likelihood estimate of a GGM whose missing edges are fixed at zero. Any solution must satisfy the problem’s optimality (Karush–Kuhn–Tucker) conditions: the implied covariance reproduces the observed correlation on every retained edge and on the diagonal, and the precision is zero on every removed edge. The amount by which a fitted network violates them is its certificate; a value near zero shows that the network is the optimum of its objective.

certificate(stepwise_net)
#>   method  certificate kind certified
#> 1    ggm 4.440892e-16  kkt      TRUE

The violation is at the level of machine precision (below \(10^{-14}\)), so the reported edges are the exact maximum-likelihood estimates for the selected graph. The certificate checks the computation, not the choice of the graph or the sampling uncertainty of its edges.

How does the selection differ from the graphical lasso?

The graphical lasso, method = "glasso", estimates the same kind of network with a penalty. Fitting it to the same data shows what the penalty changes.

glasso_net <- psychnet(data = SRL_GPT, method = "glasso")
summary(glasso_net)
#> <psychnet> glasso network
#>   nodes: 5   edges: 10   (undirected)
#>   lambda: 0.00861   gamma: 0.5
#>   optimality (KKT residual): 2.21e-10
#>   edge weight: range [-0.350, 0.412], mean 0.174
as.data.frame(glasso_net)
#>    from to      weight
#> 1   CSU IV  0.41166866
#> 2   CSU SE  0.38290223
#> 3    IV SE  0.15992605
#> 4   CSU SR  0.35481359
#> 5    IV SR  0.31909443
#> 6    SE SR  0.20767208
#> 7   CSU TA  0.04906668
#> 8    IV TA  0.11058735
#> 9    SE TA  0.09871818
#> 10   SR TA -0.34985157

The graphical lasso keeps all 10 edges, including a weak CSU–TA edge (0.049) that the stepwise selection removes. Seven of the nine shared edges are larger in absolute value in the stepwise model because they are not shrunk: CSU–IV is 0.422 against 0.412, and SR–TA is -0.370 against -0.350. The two methods agree on the structure and differ in the smallest edge and in the size of the others. The stepwise estimates suit an interpretation of edge magnitudes; the graphical lasso is the safer choice when the sample is small relative to the number of variables.

Which constructs are most connected?

net_centralities() returns strength, the sum of absolute edge weights, and expected influence, the signed sum.

net_centralities(stepwise_net)
#>   node  strength expected_influence
#> 1  CSU 1.1523235         1.15232352
#> 2   IV 1.0531117         1.05311170
#> 3   SE 0.8880693         0.88806926
#> 4   SR 1.2592340         0.51929333
#> 5   TA 0.6633854        -0.07655525

Self-regulation has the largest strength (1.259), followed by cognitive strategy use (1.152). The negative edge with test anxiety reduces the expected influence of self-regulation to 0.519, whereas cognitive strategy use, with only positive edges, has equal strength and expected influence. Test anxiety has the smallest strength (0.663) and a slightly negative expected influence (-0.077). The ordering is the same as in the graphical lasso. Centrality describes the estimated graph and does not identify causes or intervention targets.

How well is each construct predicted by the others?

net_predict() reports each node’s \(R^2\), the share of its variance explained by the other nodes, computed from the precision matrix as \(R_j^2 = 1 - 1/(K_{jj}S_{jj})\).

net_predict(stepwise_net)
#>   node     type metric predictability accuracy
#> 1  CSU gaussian     R2      0.8327378       NA
#> 2   IV gaussian     R2      0.7831430       NA
#> 3   SE gaussian     R2      0.7279821       NA
#> 4   SR gaussian     R2      0.7920188       NA
#> 5   TA gaussian     R2      0.1656566       NA

The other constructs explain 83% of the variance in cognitive strategy use (\(R^2 = 0.833\)), 79% in self-regulation, 78% in intrinsic value and 73% in self-efficacy, but only 17% in test anxiety (\(R^2 = 0.166\)). These values are slightly higher than under the graphical lasso because the unshrunk edges fit the sample more closely. They are in-sample values; prediction of new respondents requires a validation sample.

What does the network look like?

cograph::splot() draws the network, here with green positive and red negative edges and a ring showing each node’s predictability.

cograph::splot(stepwise_net, psych_styling = TRUE, predictability = TRUE)

The figure shows the positive cluster, the single negative edge, and the small ring of test anxiety. Node positions come from a layout algorithm and carry no statistical meaning.

Does the selection depend on the correlation estimator?

Refitting with cor_method = "spearman" bases the selection on rank correlations, which are less affected by skew and extreme values.

spearman_net <- psychnet(data = SRL_GPT, method = "ggm",
                         cor_method = "spearman")
summary(spearman_net)
#> <psychnet> ggm network
#>   nodes: 5   edges: 7   (undirected)
#>   optimality (KKT residual): 1.11e-15
#>   edge weight: range [-0.266, 0.483], mean 0.245
as.data.frame(spearman_net)
#>   from to     weight
#> 1  CSU IV  0.4626441
#> 2  CSU SE  0.4832486
#> 3  CSU SR  0.2856316
#> 4   IV SR  0.3865529
#> 5   SE SR  0.1948044
#> 6   IV TA  0.1652304
#> 7   SR TA -0.2664620
certificate(spearman_net)
#>   method  certificate kind certified
#> 1    ggm 1.110223e-15  kkt      TRUE

The rank-based selection keeps 7 edges instead of 9. The edges among the learning constructs, IV–TA and the negative SR–TA edge (-0.266) remain, and the IV–SE and SE–TA edges are removed. Its certificate is again at machine precision. The core structure, a positive learning cluster and a negative link from self-regulation to test anxiety, does not depend on the estimator; the weaker edges do, and they should be reported with that caveat.

Reporting the analysis

A report states the variables, the sample size, the correlation estimator, the treatment of missing data, the selection criterion with its \(\gamma\), and that the edges are unpenalized maximum-likelihood partial correlations of the selected graph.

In this analysis, stepwise selection with the BIC (\(\gamma = 0\)) on Pearson correlations of 300 complete responses selected 9 edges. The largest positive partial correlation joined cognitive strategy use and intrinsic value (0.422), and the only negative one joined self-regulation and test anxiety (-0.370). A rank-based selection kept 7 edges with the same core structure.

Assumptions and when to use stepwise selection

References

Epskamp, S., & Fried, E. I. (2018). A tutorial on regularized partial correlation networks. Psychological Methods, 23(4), 617-634. doi:10.1037/met0000167

Foygel, R., & Drton, M. (2010). Extended Bayesian information criteria for Gaussian graphical models. Advances in Neural Information Processing Systems, 23, 604-612.

Saqr, M. (2025). Delving into the psychology of machines: Exploring the structure of self-regulated learning via LLM-generated survey responses. Computers in Human Behavior, 173, 108769. doi:10.1016/j.chb.2025.108769

Williams, D. R., Rhemtulla, M., Wysocki, A. C., & Rast, P. (2019). On nonregularized estimation of psychological networks. Multivariate Behavioral Research, 54(5), 719-750. doi:10.1080/00273171.2019.1575716