Accessor functions for the outputs slots in a simList object.
If a module saves a file to disk during events, it can be useful to keep track
of the files that are saved e.g., for saveSimList() so that all files can
be added to the archive. In addition to setting outputs at the simInit
stage, a module developer can also put this in a using any saving mechanism that
is relevant (e.g., qs2::qs_save, saveRDS etc.). When a module event does this
it can be useful to register that saved file. registerOutputs offers an additional
mechanism to do this. See examples.
outputs(sim)
# S4 method for class 'simList'
outputs(sim)
outputs(sim) <- value
# S4 method for class 'simList'
outputs(sim) <- value
registerOutputs(filename, sim, ...)
outputArgs(sim)
# S4 method for class 'simList'
outputArgs(sim)
outputArgs(sim) <- value
# S4 method for class 'simList'
outputArgs(sim) <- valueA simList. If missing, then the function will search in the call
stack, so it will find it if it is in a SpaDES module.
The object to be stored at the slot. See Details.
The filename to register in the outputs(sim) data.frame. If
missing, an attempt will be made to search for either a file or filename
argument in the call itself. This means that this function can be used with
the pipe, as long as the returned return from the upstream pipe function is
a filename or if it is NULL (e.g., saveRDS), then it will find the file
argument and use that.
Not used.
A simList which will be the sim passed in with a new object registered
in the outputs(sim)
These functions are one of three mechanisms to add information about which output files to save.
As arguments to a simInit call. Specifically, inputs or outputs.
See ?simInit.
With the outputs(simList) function call.
By adding a function called .inputObjects inside a module, which will be executed
during the simInit call. This last way is the most "modular" way to create
default data sets for your model.
See below for more details.
Note using registerOutputs: a user can pass any other
arguments to registerOutputs that are in the
outputs(sim) data.frame, such as objectName, fun, package, though these
will not be used to save the files as this function is only about
registering an output that has already been saved.
The automatic file type handling only adds the correct extension from a given
fun and package. It does not do the inverse, from a given extension find the
correct fun and package.
simInitoutputs accepts a data.frame similar to the inputs data.frame, but
with up to 7 columns.
objectName | required, character string indicating the name of the object
in the simList that will be saved to disk (without the sim$ prefix). |
file | optional, a character string indicating the file path to save to.
The default is to concatenate objectName with the model timeunit and
saveTime, separated by underscore, '_'. So a default filename would be
"Fires_year1.rds". |
fun | optional, a character string indicating the function to use to
save that file. The default is saveRDS() |
package | optional character string indicating the package in
which to find the fun); |
saveTime | optional numeric, indicating when in simulation time the file
should be saved. The default is the lowest priority at end(sim),
i.e., at the very end. |
arguments | optional list of lists of named arguments, one list for each
fun. For example, if fun = "write.csv",
arguments = list(row.names = TRUE) will pass the argument
row.names = TRUE to write.csv If there is only one list,
then it is assumed to apply to all files and will be recycled as per normal R
rules of recycling for each fun. |
eventPriority | optional numeric setting the priority given to saving the
specified object. If not set, it defaults to .last(). If set to a low value,
e.g., 0, 1, 2 and saveTime is start(sim), it should give "initial conditions"
at saveTime. |
See the modules vignette for more details (browseVignettes("SpaDES.core")).
registerOutputs() which enables files that are saved to be added to
the simList using the outputs(sim) mechanism, so the files that are saved
during a module event can be tracked at the simList level. saveSimList()
which will optionally add all the outputs that are tracked into an archive.
Plots(), outputs()
#######################
# outputs
#######################
tmpdir <- file.path(tempdir(), "outputs") |> checkPath(create = TRUE)
tmpFile <- file.path(tmpdir, "temp.rds")
tempObj <- 1:10
# Can add data.frame of outputs directly into simInit call
sim <- simInit(objects = c("tempObj"),
outputs = data.frame(objectName = "tempObj"),
paths = list(outputPath = tmpdir))
#> Setting:
#> options(
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> )
#> Paths set to:
#> options(
#> rasterTmpDir = '/tmp/Rtmp4BAZIc/scratch/raster'
#> reproducible.cachePath = '/tmp/Rtmp4BAZIc/cache'
#> spades.inputPath = '/tmp/Rtmp4BAZIc/inputs'
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> spades.modulePath = '/tmp/Rtmp4BAZIc/sampleModules'
#> spades.scratchPath = '/tmp/Rtmp4BAZIc/scratch'
#> )
#> terra::terraOptions(tempdir = '/tmp/Rtmp4BAZIc/scratch/terra')
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 simInit User-supplied objects passed into sim for spades call:
#> May20 20:31:54 simInit <char>
#> May20 20:31:54 simInit 1: tempObj
#> Elapsed time for simInit: 0.03511786 secs
outputs(sim) # To see what will be saved, when, what filename
#> objectName file fun package saveTime saved arguments
#> 1 tempObj /tmp/Rtmp4BAZIc/outputs/tempObj_year10.rds saveRDS base 10 NA NA
sim <- spades(sim)
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 chckpn:init total elpsd: 0.036 secs | 0 checkpoint init 0
#> May20 20:31:54 save :init total elpsd: 0.037 secs | 0 save init 0
#> May20 20:31:54 prgrss:init total elpsd: 0.039 secs | 0 progress init 0
#> May20 20:31:54 load :init total elpsd: 0.04 secs | 0 load init 0
#> May20 20:31:54 save :spades total elpsd: 0.041 secs | 10 save spades 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(sim) # To see that it was saved, when, what filename
#> objectName file fun package saveTime saved arguments
#> 1 tempObj /tmp/Rtmp4BAZIc/outputs/tempObj_year10.rds saveRDS base 10 TRUE NA
# Also can add using assignment after a simList object has been made
sim <- simInit(objects = c("tempObj"), paths = list(outputPath = tmpdir))
#> Setting:
#> options(
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> )
#> Paths set to:
#> options(
#> rasterTmpDir = '/tmp/Rtmp4BAZIc/scratch/raster'
#> reproducible.cachePath = '/tmp/Rtmp4BAZIc/cache'
#> spades.inputPath = '/tmp/Rtmp4BAZIc/inputs'
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> spades.modulePath = '/tmp/Rtmp4BAZIc/sampleModules'
#> spades.scratchPath = '/tmp/Rtmp4BAZIc/scratch'
#> )
#> terra::terraOptions(tempdir = '/tmp/Rtmp4BAZIc/scratch/terra')
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 simInit User-supplied objects passed into sim for spades call:
#> May20 20:31:54 simInit <char>
#> May20 20:31:54 simInit 1: tempObj
#> Elapsed time for simInit: 0.03115892 secs
outputs(sim) <- data.frame(objectName = "tempObj", saveTime = 1:10)
sim <- spades(sim)
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 chckpn:init total elpsd: 0.032 secs | 0 checkpoint init 0
#> May20 20:31:54 save :init total elpsd: 0.034 secs | 0 save init 0
#> May20 20:31:54 prgrss:init total elpsd: 0.035 secs | 0 progress init 0
#> May20 20:31:54 load :init total elpsd: 0.037 secs | 0 load init 0
#> May20 20:31:54 save :spades total elpsd: 0.038 secs | 1 save spades 10
#> May20 20:31:54 save :later total elpsd: 0.04 secs | 2 save later 10
#> May20 20:31:54 save :later total elpsd: 0.042 secs | 3 save later 10
#> May20 20:31:54 save :later total elpsd: 0.044 secs | 4 save later 10
#> May20 20:31:54 save :later total elpsd: 0.045 secs | 5 save later 10
#> May20 20:31:54 save :later total elpsd: 0.047 secs | 6 save later 10
#> May20 20:31:54 save :later total elpsd: 0.049 secs | 7 save later 10
#> May20 20:31:54 save :later total elpsd: 0.05 secs | 8 save later 10
#> May20 20:31:54 save :later total elpsd: 0.052 secs | 9 save later 10
#> May20 20:31:54 save :later total elpsd: 0.054 secs | 10 save later 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(sim) # To see that it was saved, when, what filename.
#> objectName saveTime file fun package saved arguments
#> 1 tempObj 1 /tmp/Rtmp4BAZIc/outputs/tempObj_year01.rds saveRDS base TRUE NA
#> 2 tempObj 2 /tmp/Rtmp4BAZIc/outputs/tempObj_year02.rds saveRDS base TRUE NA
#> 3 tempObj 3 /tmp/Rtmp4BAZIc/outputs/tempObj_year03.rds saveRDS base TRUE NA
#> 4 tempObj 4 /tmp/Rtmp4BAZIc/outputs/tempObj_year04.rds saveRDS base TRUE NA
#> 5 tempObj 5 /tmp/Rtmp4BAZIc/outputs/tempObj_year05.rds saveRDS base TRUE NA
#> 6 tempObj 6 /tmp/Rtmp4BAZIc/outputs/tempObj_year06.rds saveRDS base TRUE NA
#> 7 tempObj 7 /tmp/Rtmp4BAZIc/outputs/tempObj_year07.rds saveRDS base TRUE NA
#> 8 tempObj 8 /tmp/Rtmp4BAZIc/outputs/tempObj_year08.rds saveRDS base TRUE NA
#> 9 tempObj 9 /tmp/Rtmp4BAZIc/outputs/tempObj_year09.rds saveRDS base TRUE NA
#> 10 tempObj 10 /tmp/Rtmp4BAZIc/outputs/tempObj_year10.rds saveRDS base TRUE NA
# can do highly variable saving
tempObj2 <- paste("val", 1:10)
df1 <- data.frame(col1 = tempObj, col2 = tempObj2)
sim <- simInit(objects = c("tempObj", "tempObj2", "df1"),
paths = list(outputPath = tmpdir))
#> Setting:
#> options(
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> )
#> Paths set to:
#> options(
#> rasterTmpDir = '/tmp/Rtmp4BAZIc/scratch/raster'
#> reproducible.cachePath = '/tmp/Rtmp4BAZIc/cache'
#> spades.inputPath = '/tmp/Rtmp4BAZIc/inputs'
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> spades.modulePath = '/tmp/Rtmp4BAZIc/sampleModules'
#> spades.scratchPath = '/tmp/Rtmp4BAZIc/scratch'
#> )
#> terra::terraOptions(tempdir = '/tmp/Rtmp4BAZIc/scratch/terra')
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 simInit User-supplied objects passed into sim for spades call:
#> May20 20:31:54 simInit <char>
#> May20 20:31:54 simInit 1: df1
#> May20 20:31:54 simInit 2: tempObj
#> May20 20:31:54 simInit 3: tempObj2
#> Elapsed time for simInit: 0.03325033 secs
outputs(sim) <- data.frame(
objectName = c(rep("tempObj", 2), rep("tempObj2", 3), "df1"),
saveTime = c(c(1, 4), c(2, 6, 7), end(sim)),
fun = c(rep("saveRDS", 5), "write.csv"),
package = c(rep("base", 5), "utils"),
stringsAsFactors = FALSE)
# since write.csv has a default of adding a column, x, with rownames, must add additional
# argument for 6th row in data.frame (corresponding to the write.csv function)
outputArgs(sim)[[6]] <- list(row.names = FALSE)
sim <- spades(sim)
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 chckpn:init total elpsd: 0.034 secs | 0 checkpoint init 0
#> May20 20:31:54 save :init total elpsd: 0.036 secs | 0 save init 0
#> May20 20:31:54 prgrss:init total elpsd: 0.037 secs | 0 progress init 0
#> May20 20:31:54 load :init total elpsd: 0.039 secs | 0 load init 0
#> May20 20:31:54 save :spades total elpsd: 0.04 secs | 1 save spades 10
#> May20 20:31:54 save :later total elpsd: 0.042 secs | 2 save later 10
#> May20 20:31:54 save :later total elpsd: 0.044 secs | 4 save later 10
#> May20 20:31:54 save :later total elpsd: 0.046 secs | 6 save later 10
#> May20 20:31:54 save :later total elpsd: 0.047 secs | 7 save later 10
#> May20 20:31:54 save :later total elpsd: 0.049 secs | 10 save later 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(sim)
#> objectName saveTime fun package file saved arguments
#> 1 tempObj 1 saveRDS base /tmp/Rtmp4BAZIc/outputs/tempObj_year01.rds TRUE NA
#> 2 tempObj 4 saveRDS base /tmp/Rtmp4BAZIc/outputs/tempObj_year04.rds TRUE NA
#> 3 tempObj2 2 saveRDS base /tmp/Rtmp4BAZIc/outputs/tempObj2_year02.rds TRUE NA
#> 4 tempObj2 6 saveRDS base /tmp/Rtmp4BAZIc/outputs/tempObj2_year06.rds TRUE NA
#> 5 tempObj2 7 saveRDS base /tmp/Rtmp4BAZIc/outputs/tempObj2_year07.rds TRUE NA
#> 6 df1 10 write.csv utils /tmp/Rtmp4BAZIc/outputs/df1_year10.csv TRUE FALSE
# read one back in just to test it all worked as planned
newObj <- read.csv(dir(tmpdir, pattern = "year10.csv", full.name = TRUE))
newObj
#> col1 col2
#> 1 1 val 1
#> 2 2 val 2
#> 3 3 val 3
#> 4 4 val 4
#> 5 5 val 5
#> 6 6 val 6
#> 7 7 val 7
#> 8 8 val 8
#> 9 9 val 9
#> 10 10 val 10
# using saving with SpaDES-aware methods
# To see current ones SpaDES can do
.saveFileExtensions()
#> exts fun package
#> 1 rds saveRDS base
#> 2 qs qsave qs
#> 3 qs2 qs_save qs2
#> 6 grd writeRaster raster
#> 8 tif writeRaster terra
#> 7 shp writeVector terra
#> 5 csv write.csv utils
#> 4 txt write.table utils
library(terra)
ras <- rast(ncol = 4, nrow = 5)
ras[] <- 1:20
sim <- simInit(objects = c("ras"), paths = list(outputPath = tmpdir))
#> Setting:
#> options(
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> )
#> Paths set to:
#> options(
#> rasterTmpDir = '/tmp/Rtmp4BAZIc/scratch/raster'
#> reproducible.cachePath = '/tmp/Rtmp4BAZIc/cache'
#> spades.inputPath = '/tmp/Rtmp4BAZIc/inputs'
#> spades.outputPath = '/tmp/Rtmp4BAZIc/outputs'
#> spades.modulePath = '/tmp/Rtmp4BAZIc/sampleModules'
#> spades.scratchPath = '/tmp/Rtmp4BAZIc/scratch'
#> )
#> terra::terraOptions(tempdir = '/tmp/Rtmp4BAZIc/scratch/terra')
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 simInit User-supplied objects passed into sim for spades call:
#> May20 20:31:54 simInit <char>
#> May20 20:31:54 simInit 1: ras
#> Elapsed time for simInit: 0.03450036 secs
outputs(sim) <- data.frame(
file = "test",
fun = "writeRaster",
package = "terra",
objectName = "ras",
stringsAsFactors = FALSE)
simOut <- spades(sim)
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> May20 20:31:54 chckpn:init total elpsd: 0.035 secs | 0 checkpoint init 0
#> May20 20:31:54 save :init total elpsd: 0.037 secs | 0 save init 0
#> May20 20:31:54 prgrss:init total elpsd: 0.038 secs | 0 progress init 0
#> May20 20:31:54 load :init total elpsd: 0.04 secs | 0 load init 0
#> May20 20:31:54 save :spades total elpsd: 0.041 secs | 10 save spades 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(simOut)
#> file fun package objectName saveTime saved arguments
#> 1 /tmp/Rtmp4BAZIc/outputs/test_year10.tif writeRaster terra ras 10 TRUE NA
newRas <- rast(dir(tmpdir, full.name = TRUE, pattern = ".tif")[1])
all.equal(newRas, ras) # Should be TRUE
#> [1] TRUE
# Clean up after
unlink(tmpdir, recursive = TRUE)
# For `registerOutputs`
sim <- simInit()
#> Setting:
#> options(
#> reproducible.cachePath = '/tmp/Rtmp4BAZIc/cache'
#> spades.inputPath = '/tmp/Rtmp4BAZIc/inputs'
#> spades.outputPath = '/tmp/Rtmp4BAZIc'
#> spades.modulePath = '/tmp/Rtmp4BAZIc/sampleModules'
#> spades.scratchPath = '/tmp/Rtmp4BAZIc/scratch'
#> )
#> May20 20:31:54 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Elapsed time for simInit: 0.02592373 secs
# This would normally be a save call, e.g., `writeRaster`
tf <- reproducible::tempfile2(fileext = ".tif")
sim <- registerOutputs(sim, filename = tf)
# Using a pipe
tf <- reproducible::tempfile2(fileext = ".rds")
sim$a <- 1
sim <- saveRDS(sim$a, tf) |> registerOutputs()
# confirm:
outputs(sim) # has object --> saved = TRUE
#> file fun package objectName saveTime saved arguments
#> <char> <char> <char> <char> <num> <lgcl> <AsIs>
#> 1: /tmp/Rtmp4BAZIc/reproducible/GDtn00zZ/file2d5f5719fedc.tif <NA> <NA> <NA> 0 TRUE NA
#> 2: /tmp/Rtmp4BAZIc/reproducible/3o4mBYUC/file2d5f8f93b06.rds <NA> <NA> <NA> 0 TRUE NA