Simulation outputs

Accessor functions for the outputs slots in a simList object.

If a module saves a file to disk during events, it can be useful to keep track of the files that are saved e.g., for saveSimList() so that all files can be added to the archive. In addition to setting outputs at the simInit stage, a module developer can also put this in a using any saving mechanism that is relevant (e.g., qs::qsave, saveRDS etc.). When a module event does this it can be useful to register that saved file. registerOutputs offers an additional mechanism to do this. See examples.

outputs(sim)

# S4 method for class 'simList'
outputs(sim)

outputs(sim) <- value

# S4 method for class 'simList'
outputs(sim) <- value

registerOutputs(filename, sim, ...)

outputArgs(sim)

# S4 method for class 'simList'
outputArgs(sim)

outputArgs(sim) <- value

# S4 method for class 'simList'
outputArgs(sim) <- value

Arguments

sim: A simList. If missing, then the function will search in the call stack, so it will find it if it is in a SpaDES module.
value: The object to be stored at the slot. See Details.
filename: The filename to register in the outputs(sim) data.frame. If missing, an attempt will be made to search for either a file or filename argument in the call itself. This means that this function can be used with the pipe, as long as the returned return from the upstream pipe function is a filename or if it is NULL (e.g., saveRDS), then it will find the file argument and use that.
...: Not used.

Value

A simList which will be the sim passed in with a new object registered in the outputs(sim)

Details

These functions are one of three mechanisms to add information about which output files to save.

As arguments to a simInit call. Specifically, inputs or outputs. See ?simInit.
With the outputs(simList) function call.
By adding a function called .inputObjects inside a module, which will be executed during the simInit call. This last way is the most "modular" way to create default data sets for your model.

See below for more details.

Note using registerOutputs: a user can pass any other arguments to registerOutputs that are in the outputs(sim) data.frame, such as objectName, fun, package, though these will not be used to save the files as this function is only about registering an output that has already been saved.

Note

The automatic file type handling only adds the correct extension from a given fun and package. It does not do the inverse, from a given extension find the correct fun and package.

outputs function or argument in `simInit`

outputs accepts a data.frame similar to the inputs data.frame, but with up to 6 columns.

`objectName`	required, character string indicating the name of the object in the `simList` that will be saved to disk (without the `sim$` prefix).
`file`	optional, a character string indicating the file path to save to. The default is to concatenate `objectName` with the model timeunit and `saveTime`, separated by underscore, '`_`'. So a default filename would be `"Fires_year1.rds"`.
`fun`	optional, a character string indicating the function to use to save that file. The default is `saveRDS()`
`package`	optional character string indicating the package in which to find the `fun`);
`saveTime`	optional numeric, indicating when in simulation time the file should be saved. The default is the lowest priority at `end(sim)`, i.e., at the very end.
`arguments`	is a list of lists of named arguments, one list for each `fun`. For example, if `fun = "write.csv"`, `arguments = list(row.names = TRUE)` will pass the argument `row.names = TRUE` to `write.csv` If there is only one list, then it is assumed to apply to all files and will be recycled as per normal R rules of recycling for each `fun`.

See the modules vignette for more details (browseVignettes("SpaDES.core")).

Examples

#######################
# outputs
#######################

tmpdir <- file.path(tempdir(), "outputs") |> checkPath(create = TRUE)
tmpFile <- file.path(tmpdir, "temp.rds")
tempObj <- 1:10

# Can add data.frame of outputs directly into simInit call
sim <- simInit(objects = c("tempObj"),
               outputs = data.frame(objectName = "tempObj"),
               paths = list(outputPath = tmpdir))
#> Setting:
#>   options(
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>   )
#> Paths set to:
#>   options(
#>     rasterTmpDir = '/tmp/RtmpIQ0Aky/scratch/raster'
#>     reproducible.cachePath = '/tmp/RtmpIQ0Aky/cache'
#>     spades.inputPath = '/tmp/RtmpIQ0Aky/inputs'
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>     spades.modulePath = '/tmp/RtmpIQ0Aky/sampleModules'
#>     spades.scratchPath = '/tmp/RtmpIQ0Aky/scratch'
#>   )
#>   terra::terraOptions(tempdir = '/tmp/RtmpIQ0Aky/scratch/terra'
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Elpsed time for simInit: 0.03508925 secs
outputs(sim) # To see what will be saved, when, what filename
#>   objectName                                       file     fun package
#> 1    tempObj /tmp/RtmpIQ0Aky/outputs/tempObj_year10.rds saveRDS    base
#>   saveTime saved arguments
#> 1       10    NA        NA
sim <- spades(sim)
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Jan27 19:04:17 chckpn:init total elpsd: 0.036 secs | 0 checkpoint init 0
#> Jan27 19:04:17 save  :init total elpsd: 0.038 secs | 0 save init 0
#> Jan27 19:04:17 prgrss:init total elpsd: 0.04 secs | 0 progress init 0
#> Jan27 19:04:17 load  :init total elpsd: 0.042 secs | 0 load init 0
#> Jan27 19:04:17 save  :spades total elpsd: 0.043 secs | 10 save spades 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(sim) # To see that it was saved, when, what filename
#>   objectName                                       file     fun package
#> 1    tempObj /tmp/RtmpIQ0Aky/outputs/tempObj_year10.rds saveRDS    base
#>   saveTime saved arguments
#> 1       10  TRUE        NA

# Also can add using assignment after a simList object has been made
sim <- simInit(objects = c("tempObj"), paths = list(outputPath = tmpdir))
#> Setting:
#>   options(
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>   )
#> Paths set to:
#>   options(
#>     rasterTmpDir = '/tmp/RtmpIQ0Aky/scratch/raster'
#>     reproducible.cachePath = '/tmp/RtmpIQ0Aky/cache'
#>     spades.inputPath = '/tmp/RtmpIQ0Aky/inputs'
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>     spades.modulePath = '/tmp/RtmpIQ0Aky/sampleModules'
#>     spades.scratchPath = '/tmp/RtmpIQ0Aky/scratch'
#>   )
#>   terra::terraOptions(tempdir = '/tmp/RtmpIQ0Aky/scratch/terra'
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Elpsed time for simInit: 0.03234625 secs
outputs(sim) <- data.frame(objectName = "tempObj", saveTime = 1:10)
sim <- spades(sim)
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Jan27 19:04:17 chckpn:init total elpsd: 0.034 secs | 0 checkpoint init 0
#> Jan27 19:04:17 save  :init total elpsd: 0.036 secs | 0 save init 0
#> Jan27 19:04:17 prgrss:init total elpsd: 0.037 secs | 0 progress init 0
#> Jan27 19:04:17 load  :init total elpsd: 0.039 secs | 0 load init 0
#> Jan27 19:04:17 save  :spades total elpsd: 0.041 secs | 1 save spades 10
#> Jan27 19:04:17 save  :later total elpsd: 0.046 secs | 2 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.05 secs | 3 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.054 secs | 4 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.059 secs | 5 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.063 secs | 6 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.067 secs | 7 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.072 secs | 8 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.076 secs | 9 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.1 secs | 10 save later 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(sim) # To see that it was saved, when, what filename.
#>    objectName saveTime                                       file     fun
#> 1     tempObj        1 /tmp/RtmpIQ0Aky/outputs/tempObj_year01.rds saveRDS
#> 2     tempObj        2 /tmp/RtmpIQ0Aky/outputs/tempObj_year02.rds saveRDS
#> 3     tempObj        3 /tmp/RtmpIQ0Aky/outputs/tempObj_year03.rds saveRDS
#> 4     tempObj        4 /tmp/RtmpIQ0Aky/outputs/tempObj_year04.rds saveRDS
#> 5     tempObj        5 /tmp/RtmpIQ0Aky/outputs/tempObj_year05.rds saveRDS
#> 6     tempObj        6 /tmp/RtmpIQ0Aky/outputs/tempObj_year06.rds saveRDS
#> 7     tempObj        7 /tmp/RtmpIQ0Aky/outputs/tempObj_year07.rds saveRDS
#> 8     tempObj        8 /tmp/RtmpIQ0Aky/outputs/tempObj_year08.rds saveRDS
#> 9     tempObj        9 /tmp/RtmpIQ0Aky/outputs/tempObj_year09.rds saveRDS
#> 10    tempObj       10 /tmp/RtmpIQ0Aky/outputs/tempObj_year10.rds saveRDS
#>    package saved arguments
#> 1     base  TRUE        NA
#> 2     base  TRUE        NA
#> 3     base  TRUE        NA
#> 4     base  TRUE        NA
#> 5     base  TRUE        NA
#> 6     base  TRUE        NA
#> 7     base  TRUE        NA
#> 8     base  TRUE        NA
#> 9     base  TRUE        NA
#> 10    base  TRUE        NA

# can do highly variable saving
tempObj2 <- paste("val", 1:10)
df1 <- data.frame(col1 = tempObj, col2 = tempObj2)
sim <- simInit(objects = c("tempObj", "tempObj2", "df1"),
  paths = list(outputPath = tmpdir))
#> Setting:
#>   options(
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>   )
#> Paths set to:
#>   options(
#>     rasterTmpDir = '/tmp/RtmpIQ0Aky/scratch/raster'
#>     reproducible.cachePath = '/tmp/RtmpIQ0Aky/cache'
#>     spades.inputPath = '/tmp/RtmpIQ0Aky/inputs'
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>     spades.modulePath = '/tmp/RtmpIQ0Aky/sampleModules'
#>     spades.scratchPath = '/tmp/RtmpIQ0Aky/scratch'
#>   )
#>   terra::terraOptions(tempdir = '/tmp/RtmpIQ0Aky/scratch/terra'
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Elpsed time for simInit: 0.03320312 secs
outputs(sim) <- data.frame(
     objectName = c(rep("tempObj", 2), rep("tempObj2", 3), "df1"),
     saveTime = c(c(1, 4), c(2, 6, 7), end(sim)),
     fun = c(rep("saveRDS", 5), "write.csv"),
     package = c(rep("base", 5), "utils"),
     stringsAsFactors = FALSE)
# since write.csv has a default of adding a column, x, with rownames, must add additional
#   argument for 6th row in data.frame (corresponding to the write.csv function)
outputArgs(sim)[[6]] <- list(row.names = FALSE)
sim <- spades(sim)
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Jan27 19:04:17 chckpn:init total elpsd: 0.034 secs | 0 checkpoint init 0
#> Jan27 19:04:17 save  :init total elpsd: 0.036 secs | 0 save init 0
#> Jan27 19:04:17 prgrss:init total elpsd: 0.038 secs | 0 progress init 0
#> Jan27 19:04:17 load  :init total elpsd: 0.04 secs | 0 load init 0
#> Jan27 19:04:17 save  :spades total elpsd: 0.042 secs | 1 save spades 10
#> Jan27 19:04:17 save  :later total elpsd: 0.047 secs | 2 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.051 secs | 4 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.056 secs | 6 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.06 secs | 7 save later 10
#> Jan27 19:04:17 save  :later total elpsd: 0.065 secs | 10 save later 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(sim)
#>   objectName saveTime       fun package
#> 1    tempObj        1   saveRDS    base
#> 2    tempObj        4   saveRDS    base
#> 3   tempObj2        2   saveRDS    base
#> 4   tempObj2        6   saveRDS    base
#> 5   tempObj2        7   saveRDS    base
#> 6        df1       10 write.csv   utils
#>                                          file saved arguments
#> 1  /tmp/RtmpIQ0Aky/outputs/tempObj_year01.rds  TRUE        NA
#> 2  /tmp/RtmpIQ0Aky/outputs/tempObj_year04.rds  TRUE        NA
#> 3 /tmp/RtmpIQ0Aky/outputs/tempObj2_year02.rds  TRUE        NA
#> 4 /tmp/RtmpIQ0Aky/outputs/tempObj2_year06.rds  TRUE        NA
#> 5 /tmp/RtmpIQ0Aky/outputs/tempObj2_year07.rds  TRUE        NA
#> 6      /tmp/RtmpIQ0Aky/outputs/df1_year10.csv  TRUE     FALSE

# read one back in just to test it all worked as planned
newObj <- read.csv(dir(tmpdir, pattern = "year10.csv", full.name = TRUE))
newObj
#>    col1   col2
#> 1     1  val 1
#> 2     2  val 2
#> 3     3  val 3
#> 4     4  val 4
#> 5     5  val 5
#> 6     6  val 6
#> 7     7  val 7
#> 8     8  val 8
#> 9     9  val 9
#> 10   10 val 10

# using saving with SpaDES-aware methods
# To see current ones SpaDES can do
.saveFileExtensions()
#>   exts         fun package
#> 1  rds     saveRDS    base
#> 2   qs       qsave      qs
#> 5  grd writeRaster  raster
#> 7  tif writeRaster   terra
#> 6  shp writeVector   terra
#> 4  csv   write.csv   utils
#> 3  txt write.table   utils

library(terra)
ras <- rast(ncol = 4, nrow = 5)
ras[] <- 1:20

sim <- simInit(objects = c("ras"), paths = list(outputPath = tmpdir))
#> Setting:
#>   options(
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>   )
#> Paths set to:
#>   options(
#>     rasterTmpDir = '/tmp/RtmpIQ0Aky/scratch/raster'
#>     reproducible.cachePath = '/tmp/RtmpIQ0Aky/cache'
#>     spades.inputPath = '/tmp/RtmpIQ0Aky/inputs'
#>     spades.outputPath = '/tmp/RtmpIQ0Aky/outputs'
#>     spades.modulePath = '/tmp/RtmpIQ0Aky/sampleModules'
#>     spades.scratchPath = '/tmp/RtmpIQ0Aky/scratch'
#>   )
#>   terra::terraOptions(tempdir = '/tmp/RtmpIQ0Aky/scratch/terra'
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Elpsed time for simInit: 0.03382111 secs
outputs(sim) <- data.frame(
  file = "test",
  fun = "writeRaster",
  package = "terra",
  objectName = "ras",
  stringsAsFactors = FALSE)

simOut <- spades(sim)
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Jan27 19:04:17 chckpn:init total elpsd: 0.035 secs | 0 checkpoint init 0
#> Jan27 19:04:17 save  :init total elpsd: 0.037 secs | 0 save init 0
#> Jan27 19:04:17 prgrss:init total elpsd: 0.039 secs | 0 progress init 0
#> Jan27 19:04:17 load  :init total elpsd: 0.041 secs | 0 load init 0
#> Jan27 19:04:17 save  :spades total elpsd: 0.042 secs | 10 save spades 10
#> simList saved in
#> SpaDES.core:::savedSimEnv()$.sim
#> It will be deleted at next spades() call.
outputs(simOut)
#>                                      file         fun package objectName
#> 1 /tmp/RtmpIQ0Aky/outputs/test_year10.tif writeRaster   terra        ras
#>   saveTime saved arguments
#> 1       10  TRUE        NA
newRas <- rast(dir(tmpdir, full.name = TRUE, pattern = ".tif")[1])
all.equal(newRas, ras) # Should be TRUE
#> [1] TRUE
# Clean up after
unlink(tmpdir, recursive = TRUE)
# For `registerOutputs`
sim <- simInit()
#> Setting:
#>   options(
#>     reproducible.cachePath = '/tmp/RtmpIQ0Aky/cache'
#>     spades.inputPath = '/tmp/RtmpIQ0Aky/inputs'
#>     spades.outputPath = '/tmp/RtmpIQ0Aky'
#>     spades.modulePath = '/tmp/RtmpIQ0Aky/sampleModules'
#>     spades.scratchPath = '/tmp/RtmpIQ0Aky/scratch'
#>   )
#> Jan27 19:04:17 simInit Using setDTthreads(1). To change: 'options(spades.DTthreads = X)'.
#> Elpsed time for simInit: 0.02904081 secs
# This would normally be a save call, e.g., `writeRaster`
tf <- reproducible::tempfile2(fileext = ".tif")
sim <- registerOutputs(sim, filename = tf)

# Using a pipe
tf <- reproducible::tempfile2(fileext = ".rds")
sim$a <- 1
sim <- saveRDS(sim$a, tf) |> registerOutputs()
# confirm:
outputs(sim) # has object --> saved = TRUE
#>                                                          file    fun package
#>                                                        <char> <char>  <char>
#> 1:  /tmp/RtmpIQ0Aky/reproducible/EcNT7Kk8/fileaf9b9aee561.tif   <NA>    <NA>
#> 2: /tmp/RtmpIQ0Aky/reproducible/mxjF4MFs/fileaf9b137ae80a.rds   <NA>    <NA>
#>    objectName saveTime  saved arguments
#>        <char>    <num> <lgcl>    <AsIs>
#> 1:       <NA>        0   TRUE        NA
#> 2:       <NA>        0   TRUE        NA