Last updated on 2026-04-18 17:51:47 CEST.
| Package | ERROR | NOTE | OK |
|---|---|---|---|
| miesmuschel | 14 | ||
| mlr | 2 | 12 | |
| mlr3pipelines | 3 | 11 | |
| mlrMBO | 14 | ||
| paradox | 14 | ||
| ParamHelpers | 14 |
Current CRAN status: OK: 14
Current CRAN status: NOTE: 2, OK: 12
Version: 2.19.3
Check: package dependencies
Result: NOTE
Package suggested but not available for checking: ‘Rmpi’
Flavors: r-oldrel-macos-arm64, r-oldrel-macos-x86_64
Version: 2.19.3
Check: installed package size
Result: NOTE
installed size is 5.6Mb
sub-directories of 1Mb or more:
R 2.0Mb
data 2.3Mb
Flavors: r-oldrel-macos-arm64, r-oldrel-macos-x86_64
Current CRAN status: ERROR: 3, OK: 11
Version: 0.11.0
Check: examples
Result: ERROR
Running examples in ‘mlr3pipelines-Ex.R’ failed
The error most likely occurred in:
> ### Name: mlr_pipeops_nmf
> ### Title: Non-negative Matrix Factorization
> ### Aliases: mlr_pipeops_nmf PipeOpNMF
>
> ### ** Examples
>
> ## Don't show:
> if (mlr3misc::require_namespaces(c("NMF", "MASS"), quietly = TRUE)) withAutoprint({ # examplesIf
+ ## End(Don't show)
+ ## Don't show:
+ # NMF attaches these packages to search path on load, #929
+ lapply(c("package:Biobase", "package:BiocGenerics", "package:generics"), detach, character.only = TRUE)
+ ## End(Don't show)
+ library("mlr3")
+
+ task = tsk("iris")
+ pop = po("nmf")
+
+ task$data()
+ pop$train(list(task))[[1]]$data()
+
+ pop$state
+ ## Don't show:
+ # BiocGenerics overwrites printer for our tables mlr-org/mlr3#1112
+ # Necessary as detaching packages does not remove registered S3 methods
+ suppressWarnings(try(rm("format.list", envir = .BaseNamespaceEnv$.__S3MethodsTable__.), silent = TRUE))
+ ## End(Don't show)
+ ## Don't show:
+ }) # examplesIf
> lapply(c("package:Biobase", "package:BiocGenerics", "package:generics"),
+ detach, character.only = TRUE)
Error in FUN(X[[i]], ...) : invalid 'name' argument
Calls: withAutoprint ... withVisible -> eval -> eval -> lapply -> lapply -> FUN
Execution halted
Flavors: r-oldrel-macos-arm64, r-oldrel-macos-x86_64, r-oldrel-windows-x86_64
Version: 0.11.0
Check: tests
Result: ERROR
Running ‘testthat.R’ [89s/44s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> if (requireNamespace("testthat", quietly = TRUE)) {
+ library("checkmate")
+ library("testthat")
+ library("mlr3")
+ library("paradox")
+ library("mlr3pipelines")
+ test_check("mlr3pipelines")
+ }
Starting 2 test processes.
> test_Graph.R: Training debug.multi with input list(input_1 = 1, input_2 = 1)
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R:
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_multiplicities.R:
> test_multiplicities.R:
> test_multiplicities.R: [[1]]
> test_multiplicities.R: [1] 0
> test_multiplicities.R:
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
Saving _problems/test_pipeop_datefeatures-7.R
Saving _problems/test_pipeop_datefeatures-17.R
> test_pipeop_isomap.R: 2026-03-23 08:14:28.445544: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.445781: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.448718: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.454427: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:28.466704: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.466827: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.469401: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.475139: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:28.481901: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.48208: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.486462: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.501297: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:28.501677: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:28.508111: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.508253: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.554855: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.569702: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:28.570263: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:28.599514: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.599669: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.604706: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.638057: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:28.648015: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.648238: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.656088: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.732572: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:28.733599: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:28.782767: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.782909: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.785274: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.791339: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:28.798693: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.798837: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.803385: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.818232: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:28.818598: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:28.853303: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.853446: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.855929: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.862323: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:28.87601: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.876214: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.881248: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.895996: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:28.896374: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:28.928906: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.929028: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.931067: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.936888: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:28.947236: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.947393: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.951978: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.966932: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:28.967299: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:28.985915: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:28.986056: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:28.988403: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:28.994435: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:29.007121: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:29.007337: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:29.011907: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:29.026945: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:29.027341: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:29.057147: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:29.057385: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:29.059752: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:29.065931: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:29.081954: L-Isomap embed START
> test_pipeop_isomap.R: 2026-03-23 08:14:29.082164: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:29.087034: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:29.101948: embedding
> test_pipeop_isomap.R: 2026-03-23 08:14:29.102314: DONE
> test_pipeop_isomap.R: 2026-03-23 08:14:29.123791: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:29.12392: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:29.126403: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:29.132599: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:29.152912: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:29.153068: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:29.155645: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:29.161717: Classical Scaling
> test_pipeop_isomap.R: 2026-03-23 08:14:29.167856: Isomap START
> test_pipeop_isomap.R: 2026-03-23 08:14:29.167972: constructing knn graph
> test_pipeop_isomap.R: 2026-03-23 08:14:29.170495: calculating geodesic distances
> test_pipeop_isomap.R: 2026-03-23 08:14:29.176557: Classical Scaling
Saving _problems/test_pipeop_nmf-45.R
Saving _problems/test_pipeop_nmf-73.R
Saving _problems/test_pipeop_nmf-93.R
Saving _problems/test_pipeop_nmf-98.R
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
[ FAIL 6 | WARN 0 | SKIP 128 | PASS 8754 ]
══ Skipped tests (128) ═════════════════════════════════════════════════════════
• On CRAN (125): 'test_CnfFormula_simplify.R:6:3', 'test_CnfFormula.R:591:3',
'test_Graph.R:283:3', 'test_PipeOp.R:32:1', 'test_GraphLearner.R:5:3',
'test_GraphLearner.R:221:3', 'test_GraphLearner.R:343:3',
'test_GraphLearner.R:408:3', 'test_GraphLearner.R:571:3',
'test_doublearrow.R:2:1', 'test_gunion.R:2:1',
'test_learner_weightedaverage.R:5:3', 'test_learner_weightedaverage.R:57:3',
'test_learner_weightedaverage.R:105:3',
'test_learner_weightedaverage.R:152:3', 'test_meta.R:39:3',
'test_dictionary.R:7:3', 'test_mlr_graphs_branching.R:26:3',
'test_mlr_graphs_bagging.R:6:3', 'test_mlr_graphs_robustify.R:5:3',
'test_pipeop_adas.R:8:3', 'test_pipeop_blsmote.R:8:3',
'test_pipeop_branch.R:4:3', 'test_pipeop_chunk.R:4:3',
'test_pipeop_classbalancing.R:7:3', 'test_pipeop_boxcox.R:7:3',
'test_pipeop_classweights.R:10:3', 'test_pipeop_classweightsex.R:9:3',
'test_pipeop_colapply.R:9:3', 'test_pipeop_collapsefactors.R:6:3',
'test_pipeop_copy.R:5:3', 'test_pipeop_colroles.R:6:3',
'test_pipeop_decode.R:14:3', 'test_pipeop_encode.R:21:3',
'test_pipeop_encodeimpact.R:11:3', 'test_pipeop_encodepl.R:5:3',
'test_pipeop_encodepl.R:72:3', 'test_pipeop_ensemble.R:3:1',
'test_pipeop_encodelmer.R:15:3', 'test_pipeop_encodelmer.R:37:3',
'test_pipeop_encodelmer.R:80:3', 'test_pipeop_filter.R:7:3',
'test_pipeop_fixfactors.R:9:3', 'test_pipeop_histbin.R:7:3',
'test_pipeop_featureunion.R:9:3', 'test_pipeop_featureunion.R:134:3',
'test_pipeop_ica.R:7:3', 'test_pipeop_impute.R:4:3',
'test_pipeop_info.R:3:1', 'test_pipeop_imputelearner.R:43:3',
'test_pipeop_kernelpca.R:9:3', 'test_pipeop_isomap.R:10:3',
'test_pipeop_learner.R:17:3', 'test_pipeop_learnerpicvplus.R:2:1',
'test_pipeop_missind.R:6:3', 'test_pipeop_modelmatrix.R:7:3',
'test_pipeop_multiplicityexply.R:9:3', 'test_pipeop_multiplicityimply.R:9:3',
'test_pipeop_mutate.R:9:3', 'test_pipeop_nearmiss.R:7:3',
'test_pipeop_learnercv.R:3:3', 'test_pipeop_learnercv.R:43:3',
'test_pipeop_learnercv.R:73:3', 'test_pipeop_learnercv.R:92:3',
'test_pipeop_learnercv.R:141:3', 'test_pipeop_learnercv.R:157:3',
'test_pipeop_learnercv.R:203:3', 'test_pipeop_learnercv.R:249:3',
'test_pipeop_learnercv.R:278:3', 'test_pipeop_learnercv.R:332:3',
'test_pipeop_learnercv.R:359:3', 'test_pipeop_learnercv.R:389:3',
'test_pipeop_learnercv.R:399:3', 'test_pipeop_learnercv.R:432:3',
'test_pipeop_learnercv.R:472:3', 'test_pipeop_learnercv.R:481:3',
'test_pipeop_learnercv.R:498:3', 'test_pipeop_learnercv.R:506:3',
'test_pipeop_learnercv.R:530:3', 'test_pipeop_learnercv.R:554:3',
'test_pipeop_learnercv.R:634:3', 'test_pipeop_learnercv.R:654:3',
'test_pipeop_learnercv.R:669:3', 'test_pipeop_learnercv.R:754:3',
'test_pipeop_learnercv.R:799:3', 'test_pipeop_learnercv.R:827:3',
'test_pipeop_ovr.R:9:3', 'test_pipeop_ovr.R:48:3', 'test_pipeop_pca.R:8:3',
'test_pipeop_proxy.R:2:1', 'test_pipeop_quantilebin.R:5:3',
'test_pipeop_randomprojection.R:6:3', 'test_pipeop_randomresponse.R:5:3',
'test_pipeop_removeconstants.R:6:3', 'test_pipeop_renamecolumns.R:6:3',
'test_pipeop_replicate.R:9:3', 'test_pipeop_rowapply.R:6:3',
'test_pipeop_scale.R:6:3', 'test_pipeop_scale.R:10:3',
'test_pipeop_scalemaxabs.R:6:3', 'test_pipeop_scalerange.R:7:3',
'test_pipeop_select.R:9:3', 'test_pipeop_smote.R:10:3',
'test_pipeop_smotenc.R:8:3', 'test_pipeop_spatialsign.R:3:1',
'test_pipeop_splines.R:3:1', 'test_pipeop_nmf.R:6:3',
'test_pipeop_subsample.R:6:3', 'test_pipeop_targetinvert.R:4:3',
'test_pipeop_targettrafo.R:4:3', 'test_pipeop_targetmutate.R:5:3',
'test_pipeop_targettrafoscalerange.R:5:3', 'test_pipeop_task_preproc.R:4:3',
'test_pipeop_task_preproc.R:14:3', 'test_pipeop_tomek.R:7:3',
'test_pipeop_textvectorizer.R:37:3', 'test_pipeop_textvectorizer.R:186:3',
'test_pipeop_unbranch.R:10:3', 'test_pipeop_updatetarget.R:89:3',
'test_pipeop_vtreat.R:9:3', 'test_pipeop_yeojohnson.R:7:3',
'test_ppl.R:63:3', 'test_pipeop_tunethreshold.R:111:3',
'test_pipeop_tunethreshold.R:191:3', 'test_typecheck.R:188:3'
• Skipping (1): 'test_GraphLearner.R:1278:3'
• empty test (2): 'test_pipeop_isomap.R:111:1', 'test_pipeop_missind.R:101:1'
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_pipeop_datefeatures.R:7:3'): PipeOpDateFeatures - basic properties ──
Error in `seq.Date(as.Date("2020-01-31"), length.out = 150L)`: exactly two of 'to', 'by' and 'length.out' / 'along.with' must be specified
Backtrace:
▆
1. ├─base::seq(as.Date("2020-01-31"), length.out = 150L) at test_pipeop_datefeatures.R:7:3
2. └─base::seq.Date(as.Date("2020-01-31"), length.out = 150L)
── Error ('test_pipeop_datefeatures.R:17:3'): PipeOpDateFeatures - finds POSIXct column ──
Error in `seq.Date(as.Date("2020-01-31"), length.out = 150L)`: exactly two of 'to', 'by' and 'length.out' / 'along.with' must be specified
Backtrace:
▆
1. ├─base::seq(as.Date("2020-01-31"), length.out = 150L) at test_pipeop_datefeatures.R:17:3
2. └─base::seq.Date(as.Date("2020-01-31"), length.out = 150L)
── Error ('test_pipeop_nmf.R:45:3'): PipeOpNMF - does not modify search path when NMF is not loaded, fix for #929 ──
Error in `detach(package:generics)`: invalid 'name' argument
Backtrace:
▆
1. └─base::detach(package:generics) at test_pipeop_nmf.R:45:3
── Failure ('test_pipeop_nmf.R:73:3'): PipeOpNMF - does not modify search path when NMF is loaded, fix for #929 ──
Expected `all(...)` to be TRUE.
Differences:
`actual`: FALSE
`expected`: TRUE
── Failure ('test_pipeop_nmf.R:93:3'): PipeOpNMF - does not modify search path when some of NMF's dependencies are loaded, fix for #929 ──
Expected `all(paste0("package:", c("BiocGenerics", "generics")) %in% search())` to be TRUE.
Differences:
`actual`: FALSE
`expected`: TRUE
── Error ('test_pipeop_nmf.R:98:3'): PipeOpNMF - does not modify search path when some of NMF's dependencies are loaded, fix for #929 ──
Error in `FUN(X[[i]], ...)`: invalid 'name' argument
This happened in PipeOp nmf's $train()
Backtrace:
▆
1. ├─op$train(list(tsk("iris"))) at test_pipeop_nmf.R:98:3
2. │ └─mlr3pipelines:::.__PipeOp__train(...)
3. │ ├─base::withCallingHandlers(...)
4. │ └─private$.train(input)
5. │ └─mlr3pipelines:::.__PipeOpTaskPreproc__.train(...)
6. │ └─private$.train_task(intask)
7. │ └─mlr3pipelines:::.__PipeOpTaskPreproc__.train_task(...)
8. │ ├─data.table::as.data.table(...)
9. │ └─private$.train_dt(dt, task$levels(cols), task$truth())
10. │ └─mlr3pipelines:::.__PipeOpNMF__.train_dt(...)
11. │ └─mlr3misc::map(to_be_detached, detach, character.only = TRUE)
12. │ └─base::lapply(.x, .f, ...)
13. │ └─base (local) FUN(X[[i]], ...)
14. │ └─base::stop("invalid 'name' argument")
15. └─base::.handleSimpleError(...)
16. └─mlr3pipelines (local) h(simpleError(msg, call))
[ FAIL 6 | WARN 0 | SKIP 128 | PASS 8754 ]
Error:
! Test failures.
Execution halted
Flavor: r-oldrel-macos-arm64
Version: 0.11.0
Check: tests
Result: ERROR
Running ‘testthat.R’ [284s/278s]
Running the tests in ‘tests/testthat.R’ failed.
Complete output:
> if (requireNamespace("testthat", quietly = TRUE)) {
+ library("checkmate")
+ library("testthat")
+ library("mlr3")
+ library("paradox")
+ library("mlr3pipelines")
+ test_check("mlr3pipelines")
+ }
Starting 2 test processes.
> test_Graph.R: Training debug.multi with input list(input_1 = 1, input_2 = 1)
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_multiplicities.R:
> test_multiplicities.R: [[1]]
> test_multiplicities.R: [1] 0
> test_multiplicities.R:
> test_multiplicities.R:
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
Saving _problems/test_pipeop_datefeatures-7.R
Saving _problems/test_pipeop_datefeatures-17.R
> test_pipeop_isomap.R: 2026-04-18 10:11:55.905248: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:55.906453: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:55.926257: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:55.960667: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:56.024482: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:56.024806: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:56.032705: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:56.061227: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:56.107978: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:56.108438: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:56.125207: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:56.206676: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:56.208919: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:56.254186: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:56.254504: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:56.28799: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:56.356516: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:56.358489: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:56.532022: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:56.53242: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:56.551675: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:56.76527: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:56.80426: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:56.804691: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:56.847625: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:57.273824: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:57.278981: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:57.569228: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:57.569545: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:57.589443: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:57.631091: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:57.675: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:57.675443: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:57.698115: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:57.772551: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:57.774531: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:57.994191: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:57.994507: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:58.007433: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:58.040335: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:58.103213: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:58.103926: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:58.135696: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:58.208269: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:58.209609: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:58.308296: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:58.30874: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:58.338329: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:58.36724: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:58.430308: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:58.430832: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:58.457982: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:58.530304: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:58.531521: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:58.709334: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:58.710666: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:58.723338: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:58.747037: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:58.7967: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:58.79715: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:58.829791: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:58.901913: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:58.903142: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:58.994656: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:58.995008: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:59.003832: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:59.051768: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:59.20883: L-Isomap embed START
> test_pipeop_isomap.R: 2026-04-18 10:11:59.209327: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:59.236713: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:59.29541: embedding
> test_pipeop_isomap.R: 2026-04-18 10:11:59.297178: DONE
> test_pipeop_isomap.R: 2026-04-18 10:11:59.419245: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:59.419566: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:59.429819: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:59.453843: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:59.628986: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:59.629436: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:59.640101: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:59.673534: Classical Scaling
> test_pipeop_isomap.R: 2026-04-18 10:11:59.711224: Isomap START
> test_pipeop_isomap.R: 2026-04-18 10:11:59.711758: constructing knn graph
> test_pipeop_isomap.R: 2026-04-18 10:11:59.720044: calculating geodesic distances
> test_pipeop_isomap.R: 2026-04-18 10:11:59.754682: Classical Scaling
Saving _problems/test_pipeop_nmf-45.R
Saving _problems/test_pipeop_nmf-73.R
Saving _problems/test_pipeop_nmf-93.R
Saving _problems/test_pipeop_nmf-98.R
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
> test_pipeop_task_preproc.R: Training debug_affectcols
[ FAIL 6 | WARN 0 | SKIP 128 | PASS 8754 ]
══ Skipped tests (128) ═════════════════════════════════════════════════════════
• On CRAN (125): 'test_CnfFormula_simplify.R:6:3', 'test_CnfFormula.R:591:3',
'test_Graph.R:283:3', 'test_PipeOp.R:32:1', 'test_GraphLearner.R:5:3',
'test_GraphLearner.R:221:3', 'test_GraphLearner.R:343:3',
'test_GraphLearner.R:408:3', 'test_GraphLearner.R:571:3',
'test_doublearrow.R:2:1', 'test_gunion.R:2:1',
'test_learner_weightedaverage.R:5:3', 'test_learner_weightedaverage.R:57:3',
'test_learner_weightedaverage.R:105:3',
'test_learner_weightedaverage.R:152:3', 'test_meta.R:39:3',
'test_dictionary.R:7:3', 'test_mlr_graphs_branching.R:26:3',
'test_mlr_graphs_bagging.R:6:3', 'test_mlr_graphs_robustify.R:5:3',
'test_pipeop_adas.R:8:3', 'test_pipeop_blsmote.R:8:3',
'test_pipeop_branch.R:4:3', 'test_pipeop_chunk.R:4:3',
'test_pipeop_classbalancing.R:7:3', 'test_pipeop_boxcox.R:7:3',
'test_pipeop_classweights.R:10:3', 'test_pipeop_classweightsex.R:9:3',
'test_pipeop_collapsefactors.R:6:3', 'test_pipeop_colapply.R:9:3',
'test_pipeop_copy.R:5:3', 'test_pipeop_colroles.R:6:3',
'test_pipeop_decode.R:14:3', 'test_pipeop_encode.R:21:3',
'test_pipeop_encodeimpact.R:11:3', 'test_pipeop_encodepl.R:5:3',
'test_pipeop_encodepl.R:72:3', 'test_pipeop_ensemble.R:3:1',
'test_pipeop_encodelmer.R:15:3', 'test_pipeop_encodelmer.R:37:3',
'test_pipeop_encodelmer.R:80:3', 'test_pipeop_filter.R:7:3',
'test_pipeop_fixfactors.R:9:3', 'test_pipeop_histbin.R:7:3',
'test_pipeop_ica.R:7:3', 'test_pipeop_featureunion.R:9:3',
'test_pipeop_featureunion.R:134:3', 'test_pipeop_impute.R:4:3',
'test_pipeop_imputelearner.R:43:3', 'test_pipeop_info.R:3:1',
'test_pipeop_kernelpca.R:9:3', 'test_pipeop_isomap.R:10:3',
'test_pipeop_learner.R:17:3', 'test_pipeop_learnerpicvplus.R:2:1',
'test_pipeop_missind.R:6:3', 'test_pipeop_modelmatrix.R:7:3',
'test_pipeop_multiplicityexply.R:9:3', 'test_pipeop_multiplicityimply.R:9:3',
'test_pipeop_mutate.R:9:3', 'test_pipeop_nearmiss.R:7:3',
'test_pipeop_learnercv.R:3:3', 'test_pipeop_learnercv.R:43:3',
'test_pipeop_learnercv.R:73:3', 'test_pipeop_learnercv.R:92:3',
'test_pipeop_learnercv.R:141:3', 'test_pipeop_learnercv.R:157:3',
'test_pipeop_learnercv.R:203:3', 'test_pipeop_learnercv.R:249:3',
'test_pipeop_learnercv.R:278:3', 'test_pipeop_learnercv.R:332:3',
'test_pipeop_learnercv.R:359:3', 'test_pipeop_learnercv.R:389:3',
'test_pipeop_learnercv.R:399:3', 'test_pipeop_learnercv.R:432:3',
'test_pipeop_learnercv.R:472:3', 'test_pipeop_learnercv.R:481:3',
'test_pipeop_learnercv.R:498:3', 'test_pipeop_learnercv.R:506:3',
'test_pipeop_learnercv.R:530:3', 'test_pipeop_learnercv.R:554:3',
'test_pipeop_learnercv.R:634:3', 'test_pipeop_learnercv.R:654:3',
'test_pipeop_learnercv.R:669:3', 'test_pipeop_learnercv.R:754:3',
'test_pipeop_learnercv.R:799:3', 'test_pipeop_learnercv.R:827:3',
'test_pipeop_ovr.R:9:3', 'test_pipeop_ovr.R:48:3', 'test_pipeop_pca.R:8:3',
'test_pipeop_proxy.R:2:1', 'test_pipeop_quantilebin.R:5:3',
'test_pipeop_randomprojection.R:6:3', 'test_pipeop_randomresponse.R:5:3',
'test_pipeop_removeconstants.R:6:3', 'test_pipeop_renamecolumns.R:6:3',
'test_pipeop_replicate.R:9:3', 'test_pipeop_nmf.R:6:3',
'test_pipeop_scale.R:6:3', 'test_pipeop_scale.R:10:3',
'test_pipeop_scalemaxabs.R:6:3', 'test_pipeop_scalerange.R:7:3',
'test_pipeop_select.R:9:3', 'test_pipeop_smote.R:10:3',
'test_pipeop_rowapply.R:6:3', 'test_pipeop_spatialsign.R:3:1',
'test_pipeop_splines.R:3:1', 'test_pipeop_smotenc.R:8:3',
'test_pipeop_targetinvert.R:4:3', 'test_pipeop_targetmutate.R:5:3',
'test_pipeop_targettrafo.R:4:3', 'test_pipeop_targettrafoscalerange.R:5:3',
'test_pipeop_task_preproc.R:4:3', 'test_pipeop_task_preproc.R:14:3',
'test_pipeop_subsample.R:6:3', 'test_pipeop_tomek.R:7:3',
'test_pipeop_textvectorizer.R:37:3', 'test_pipeop_textvectorizer.R:186:3',
'test_pipeop_unbranch.R:10:3', 'test_pipeop_updatetarget.R:89:3',
'test_pipeop_vtreat.R:9:3', 'test_pipeop_yeojohnson.R:7:3',
'test_ppl.R:63:3', 'test_pipeop_tunethreshold.R:111:3',
'test_pipeop_tunethreshold.R:191:3', 'test_typecheck.R:188:3'
• Skipping (1): 'test_GraphLearner.R:1278:3'
• empty test (2): 'test_pipeop_isomap.R:111:1', 'test_pipeop_missind.R:101:1'
══ Failed tests ════════════════════════════════════════════════════════════════
── Error ('test_pipeop_datefeatures.R:7:3'): PipeOpDateFeatures - basic properties ──
Error in `seq.Date(as.Date("2020-01-31"), length.out = 150L)`: exactly two of 'to', 'by' and 'length.out' / 'along.with' must be specified
Backtrace:
▆
1. ├─base::seq(as.Date("2020-01-31"), length.out = 150L) at test_pipeop_datefeatures.R:7:3
2. └─base::seq.Date(as.Date("2020-01-31"), length.out = 150L)
── Error ('test_pipeop_datefeatures.R:17:3'): PipeOpDateFeatures - finds POSIXct column ──
Error in `seq.Date(as.Date("2020-01-31"), length.out = 150L)`: exactly two of 'to', 'by' and 'length.out' / 'along.with' must be specified
Backtrace:
▆
1. ├─base::seq(as.Date("2020-01-31"), length.out = 150L) at test_pipeop_datefeatures.R:17:3
2. └─base::seq.Date(as.Date("2020-01-31"), length.out = 150L)
── Error ('test_pipeop_nmf.R:45:3'): PipeOpNMF - does not modify search path when NMF is not loaded, fix for #929 ──
Error in `detach(package:generics)`: invalid 'name' argument
Backtrace:
▆
1. └─base::detach(package:generics) at test_pipeop_nmf.R:45:3
── Failure ('test_pipeop_nmf.R:73:3'): PipeOpNMF - does not modify search path when NMF is loaded, fix for #929 ──
Expected `all(...)` to be TRUE.
Differences:
`actual`: FALSE
`expected`: TRUE
── Failure ('test_pipeop_nmf.R:93:3'): PipeOpNMF - does not modify search path when some of NMF's dependencies are loaded, fix for #929 ──
Expected `all(paste0("package:", c("BiocGenerics", "generics")) %in% search())` to be TRUE.
Differences:
`actual`: FALSE
`expected`: TRUE
── Error ('test_pipeop_nmf.R:98:3'): PipeOpNMF - does not modify search path when some of NMF's dependencies are loaded, fix for #929 ──
Error in `FUN(X[[i]], ...)`: invalid 'name' argument
This happened in PipeOp nmf's $train()
Backtrace:
▆
1. ├─op$train(list(tsk("iris"))) at test_pipeop_nmf.R:98:3
2. │ └─mlr3pipelines:::.__PipeOp__train(...)
3. │ ├─base::withCallingHandlers(...)
4. │ └─private$.train(input)
5. │ └─mlr3pipelines:::.__PipeOpTaskPreproc__.train(...)
6. │ └─private$.train_task(intask)
7. │ └─mlr3pipelines:::.__PipeOpTaskPreproc__.train_task(...)
8. │ ├─data.table::as.data.table(...)
9. │ └─private$.train_dt(dt, task$levels(cols), task$truth())
10. │ └─mlr3pipelines:::.__PipeOpNMF__.train_dt(...)
11. │ └─mlr3misc::map(to_be_detached, detach, character.only = TRUE)
12. │ └─base::lapply(.x, .f, ...)
13. │ └─base (local) FUN(X[[i]], ...)
14. │ └─base::stop("invalid 'name' argument")
15. └─base::.handleSimpleError(...)
16. └─mlr3pipelines (local) h(simpleError(msg, call))
[ FAIL 6 | WARN 0 | SKIP 128 | PASS 8754 ]
Error:
! Test failures.
Execution halted
Flavor: r-oldrel-macos-x86_64
Version: 0.11.0
Check: tests
Result: ERROR
Running 'testthat.R' [100s]
Running the tests in 'tests/testthat.R' failed.
Complete output:
> if (requireNamespace("testthat", quietly = TRUE)) {
+ library("checkmate")
+ library("testthat")
+ library("mlr3")
+ library("paradox")
+ library("mlr3pipelines")
+ test_check("mlr3pipelines")
+ }
Starting 2 test processes.
> test_Graph.R: Training debug.multi with input list(input_1 = 1, input_2 = 1)
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_PipeOp.R: Training test_autotrain
> test_PipeOp.R: Predicting test_autotrain
> test_multiplicities.R:
> test_multiplicities.R: [[1]]
> test_multiplicities.R:
> test_multiplicities.R: [1] 0
> test_multiplicities.R:
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
> test_pipeop_blsmote.R: [1] "Borderline-SMOTE done"
Saving _problems/test_pipeop_datefeatures-7.R
Saving _problems/test_pipeop_datefeatures-17.R
> test_pipeop_histbin.R:
Error:
! testthat subprocess exited in file 'test_pipeop_histbin.R'.
Caused by error:
! R session crashed with exit code -1073741819
Backtrace:
▆
1. └─testthat::test_check("mlr3pipelines")
2. └─testthat::test_dir(...)
3. └─testthat:::test_files(...)
4. └─testthat:::test_files_parallel(...)
5. ├─withr::with_dir(...)
6. │ └─base::force(code)
7. ├─testthat::with_reporter(...)
8. │ └─base::tryCatch(...)
9. │ └─base (local) tryCatchList(expr, classes, parentenv, handlers)
10. │ └─base (local) tryCatchOne(expr, names, parentenv, handlers[[1L]])
11. │ └─base (local) doTryCatch(return(expr), name, parentenv, handler)
12. └─testthat:::parallel_event_loop_chunky(queue, reporters, ".")
13. └─queue$poll(Inf)
14. └─base::lapply(...)
15. └─testthat (local) FUN(X[[i]], ...)
16. └─private$handle_error(msg, i)
17. └─cli::cli_abort(...)
18. └─rlang::abort(...)
Execution halted
Flavor: r-oldrel-windows-x86_64
Current CRAN status: OK: 14
Current CRAN status: OK: 14
Current CRAN status: OK: 14