A connector between mlr3 and batchtools. This allows to run large-scale benchmark experiments on scheduled high-performance computing clusters.
The package comes with two core functions for switching between mlr3
and batchtools
to perform a benchmark:
design
object (as required for mlr3
’s benchmark()
function), instead of benchmark()
call batchmark()
which populates an ExperimentRegistry
for the computational jobs of the benchmark. You are now in the world of batchtools
where you can selectively submit jobs with different resources, monitor the progress or resubmit as needed.reduceResultsBatchmark()
to return to mlr3
. The resulting object is a regular BenchmarkResult
.
library("mlr3")
library("batchtools")
library("mlr3batchmark")
tasks = tsks(c("iris", "sonar"))
learners = lrns(c("classif.featureless", "classif.rpart"))
resamplings = rsmp("cv", folds = 3)
design = benchmark_grid(
tasks = tasks,
learners = learners,
resamplings = resamplings
)
reg = makeExperimentRegistry(NA)
## No readable configuration file found
## Created registry in '/tmp/Rtmp8DlMZQ/registry704553adf7a88' using cluster functions 'Interactive'
ids = batchmark(design, reg = reg)
## Adding algorithm 'run_learner'
## Adding problem 'b39ef23a66b1f1ee'
## Exporting new objects: '5ec484de3f93431b' ...
## Exporting new objects: '7c35d835f3dfae37' ...
## Exporting new objects: '70dd22724e5c724d' ...
## Adding 6 experiments ('b39ef23a66b1f1ee'[1] x 'run_learner'[2] x repls[3]) ...
## Adding problem '76c4fc7a533d41b7'
## Exporting new objects: 'b209de197d6cbe75' ...
## Adding 6 experiments ('76c4fc7a533d41b7'[1] x 'run_learner'[2] x repls[3]) ...